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(57) Abstract 

An image encoder and an image decoder 
which fecilitate the prediction of the image 
movement other tiian the parallel movement 
with a relatively small scale process, with a 
shortened processing time and with a high 
precision. An input image is divided into 
predetermined blocks and the input image is 
compressed and encoded by providing a means 
which predicts the movement compensation by 
detecting the movements between tiie frames of 
the blocks. A movement detection unit which 
includes a deformed block matching section 
which deforms only integer pixels which are 
actual sample points existing in partial regions 
corresponding to a reference image for the 
movement detection into required shapes and 
segments them and compares the deformed and 
segmented pixels with the integer pixels of the 
blocks of the input image and which outputs a 
motion vector which gives a minimum error 
which is extracted by designating the 
coordinates and a movement con4>ensation unit 
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whidi mchidcs a correspondmg point detcrmming section which specifies the coordinates of the integer pixels correspondmgly to iho blocks 
of the reference image in accordance witti the movement parameters which are obtained from the comparison ou^ut including the output 
from the deformed block matching section and deforms the putels and dctennines and which outputs a predicted partial unage are provided. 
The image deoodor is so constructed as to correspond to &e hnage encoder. 
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ENGLISH TRANSLATION FOR PCT/ JP97/00834 
SPECIFICATION 

Video Encoder, Video Decoder, Video Encoding Method, 
Video Decoding Method, and Video Encoding and Decoding 
System 

Technical Field 

The present invention relates to a highly efficient 
picture (image) encoding and decoding system for 
performing motion compensated prediction of a picture 
(image) to be encoded or decoded for encoding a 
prediction error, and for decoding reference picture 
(image) data together with the prediction error by 
referring to an encoded picture (image) . 

Background Art 

Conventional motion compensated prediction methods 
for performing highly efficient encodation of a picture 
are described below. 

The first example of a conventional motion 
compensated prediction method which will be discussed is 
a motion compensated prediction method using block 
matching that compensates for translational motion of an 
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object. For example^ in ISO/IEC 11172-2 (also known as 
the MPEG 1 video standard) / a 

forward/backward/ interpolative motion compensated 
prediction method using block matching is described. The 
second example of a conventional motion compensated 
prediction method which will be discussed is a motion 
compensated prediction method using an affine motion 
model. For example^ "Motion Compensated Prediction Using 
An Affine Motion Model" (the technical report of IE94-36, 
by the Institute of Electronics, Inf oannation and 
Communication Engineers of Japan) describes the motion 
compensated prediction method in which the displacement 
of an object in each arbitrarily shaped segment is 
modeled and expressed using affine motion parameters, and 
in which the affine motion parameters are detected so as 
to perform motion compensated prediction. 

Now, the conventional motion compensation method using 
block matching by a translational motion and the 
conventional motion compensation method using the affine 
motion model will be described in more detail below. 

Fig. 29 shows a known motion compensated prediction 
which utilizes block matching. In Fig. 29, i represents 
a position of a block on a display as a unit used for 
motion compensated prediction; fi (x, y, t) represents the 
pel value (x, y) in the block i at time t on the display; 



R represents a motion vector search range; and v 
represents a motion vector (eR) . Block matching is a 
process for detecting, within the search range R of a 
reference picture 201, a block whose pel value is most 
approximate to the pel value fi(x, y, t) of the block i 
in an input picture 202, or for detecting a pel value 
fi+v (X, y, t-1) which will minimize a prediction error 
power Dv which may be expressed in one of the following 
equations (1) • 

Dv = E {fi^(x,y, t-1) - fi<x,y,t)}2 

or 

E I fi^(x, y, t-1) - fi(x, y, t) | (1) 

The value v which minimizes Dv will be the motion 
vector. In Fig, 29, a block matching search method. using 
real sample point integer pels in a reference picture is 
referred to as an integer pel precision search, and a 
block matching search method using half-pels (interposed 
midway between the integer pels) in addition to integer 
pels is referred to as a half-pel precision search. 
Generally, under the same block matching search range, 
more search pel points can be obtained in the half-pel 
precision search than in the integer pel precision 
search. Consequently, increased prediction accuracy will 
be obtained with the half-pel precision search. 



Fig- 30 is a block diagram showing a configuration 
of a motion compensated predictor (also referred to as a 
block matching section) using a motion compensated 
prediction method in accordance with, for example, the 
MPEGI video standard. 

In the figure, reference numeral 207 is a horizontal 
displacement counter, 208 is a vertical displacement 
counter, 211 is a memory readout-address generator, 213 
is a pattern matching unit, and reference numeral 216 is 
a minimum prediction error power determinator • Reference 
numeral 203 is a horizontal displacement search range 
indication signal, 204 is a vertical displacement search 
range indication signal, 205 is input picture block data, 
206 is an input picture block position indication signal, 
209 is horizontal displacement search point data, 210 is 
vertical displacement search point data, 212 is a readout 
address, 214 is readout picture data, 215 is a prediction 
error power signal, 217 is a motion vector, 218 is a 
minimiom prediction error power signal, and 219 is a frame 
memory for storing reference picture data. 

Fig. 31 is a flow chart showing the operations of 
the conventional motion compensated predictor having the 
above-mentioned configuration of Fig. 30. 

In Fig. 31, dx represents a horizontal displacement 
search pel point; 
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dy represents a vertical displacement search pel 
point; 

range_h_min represents a lower limit in a horizontal 
displacement search range; 

range_h_max represents an upper limit in the 
horizontal displacement search range; 

range_v_min represents a Iqwer limit in a vertical 
displacement search range; 

range_v_max represents an upper limit in the 
vertical displacement search range; 

D_jain represents the minimum prediction error power; 

(x, y) are coordinates representing the position of 
a pel in a macroblock; 

DCdx, dy) represents prediction error power produced 
when dx and dy are searched; 

f (x, y) is the value of a pel (x, y) in an input 
picture macroblock; 

fr{x, y) is the value of a pel (x, y) in a reference 
picture; 

y) is a prediction error for the pel (x, y) 
when dx and dy are searched; 

MV_h is a horizontal component of a motion vector 
(indicating horizontal displacement); and 

MV_v is a vertical component of a motion vector 
(indicating vertical displacement) • 
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The block matching operation will be described in 
more detail, by referring to Figs. 43 and 44. 

1) Motion Vector Search Range Setting 

Range_h_min and range_h_jiiax are set through the 
; horizontal displacement counter 207 according to the 

horizontal displacement search range indication signal 
203. Range_v_min and range__v_max are set through the 
vertical displacement coxanter 208 according to the 
vertical displacement search range indication signal 204. 
) In addition, the initial values of dx for the horizontal 

displacement counter 207 and dy for the vertical 
displacement counter 208 are set to range_h_min and 
range^v_min, respectively. In the minimum prediction 
error power determinator 216, the minimum prediction 
3 error power D_min is set to a maximum integer value 

MAXINT (for example, OxFFFFFFFF) . These operations 
correspond to step S201 in Fig. 31. 

2) Possible Prediction picture Readout Operation 

Data on the pel (x+dx, y+dy) in a reference picture, 
) which are distant from the pel (x, y) in the input 

picture macroblock by dx and dy are fetched from the 
frame memory. The memory readout address generator 211 
illustrated in Fig. 30 receives the value of dx from the 
horizontal displacement counter 2 07 and the value of dy 
3 from the vertical displacement counter 208, and generates 



the address for the pel (x+dx, y+dy) in the frame memory. 

3) Prediction Error Power Calculation 

First, the prediction error power D(dx, dy) for the 
motion vector representing (dx, dy) is initialized to 
zero. This corresponds to step S202 in Fig. 31. The 
absolute value for the difference between the pel value - 
readout in 2) and the value of the pel (x, y) in the 
input picture macroblock is accumulated into D(dX/ dy) . 
This operation is repeated until the value of x and the 
value of y become x = y = 16. Then, the prediction error 
power D(dx, dy) produced when (dx, dy) is searched, or Dv 
given by numeral equations (1) is obtained. This 
operation is executed by the pattern matching unit 213 
illustrated in Fig. 30. Then, the pattern matching unit 
213 supplies D(dx, dy) to the minimum prediction error 
power de terminator 216 through the prediction error power 
signal 215. These operations correspond to steps S203 
through S209 in Fig. 31. 

4) Minimum Prediction Error Power Updating 

It is then determined whether the resultant D(dx, 
dy) obtained in 3) has given the minimum prediction error 
power among the searched results which have been obtained 
so far. This determination is made by the minimum 
prediction error power determinator 216 illustrated in 
Fig. 30. This corresponds to step S210 in Fig. 31. The 
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minimum prediction error power determinator 216 compares 
the value of the minimum prediction error power p_min 
therein with D(dx, dy) supplied through the prediction 
error power signal 215. If D.(dx, dy) is smaller than 
D_min, the minimum prediction error power determinator 
216 updates the value of D_min to D(dx, dy) . In 
addition, the minimum prediction error power determinator 
216 retains the values of dx and dy at that time as the 
possible motion vector (MV_h, MV_v) . This updating 
operation corresponds to step S211 in Fig. 31- 
5) Motion Vector Value Determination 

The above-mentioned 2) through 4) operations are 
repeated for all (dx, dy) within the motion vector search 
range R. (These operations correspond to steps S212 
through S215 in Fig. 31.) The final values (MV_h, MV_v) 
retained by the minimum prediction error power 
determinator 216 are output as the motion vector 217. 

Fig. 32 schematically shows a motion compensated 
prediction system in accordance with the MPEGl video 
standard. 

Under the MPEGl video standard, a motion picture 
frame is typically referred to as a picture. One picture 
is divided into macroblocks, each of which includes 16 x 
16 pels (color difference signal includes 8x8 pels) . 
For each macroblock, motion compensated prediction using 



block matching is performed. The resultant motion vector 
value and a prediction error are then encoded. 

Under the MPEGl video standard, different motion 
compensation methods can be applied to different 
individual pictures. Referring to the figure, I-pictures 
are encoded, without being subjected to motion 
compensated prediction and without reference to other 
pictures. . P-pictures are encoded using forward motion 
compensated prediction from a past encoded picture. B- 
pictures may be. encoded using forward motion compensated 
prediction from a future picture to be encoded, backward 
motion compensated prediction from a past picture, and 
interpolative prediction from the mean value between the 
past encoded picture and the future picture to be 
encoded. However, forward/backward/ interpolative motion 
compensated predictions are basically all motion 
compensated prediction using block matching which utilize 
different reference pictures for implementing the 
prediction. 

As described above, block matching has been 
established as a main method for implementing motion 
compensated prediction for current video encoding 
systems. Block matching, however, is an operation which 
determines the translational displacement of an object 
for each scjuare block such as a macroblock. Block 
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matching is based on the assumption that "a picture 
portion having the same luminance belongs to the same 
object". Consequently/ in principle, it is impossible to 
detect motions of an object other than square-block-based 
motions. For portions in which the object does not move 
according to a simple translational motion such as 
rotation, scaling up and down, zooming, or three- 
dimensional motion prediction accuracy will be reduced. 

In order to solve the above-mentioned motion 
detecting problems that are associated with the 
conventional block matching method described above, 
motion Compensated prediction using the af fine motion 
model have been proposed and aim at more accurately 
detecting the displacement of an object including 
rotation and scaling of the object as well as 
translational motion. This known solution is based on 
the assumption that (x, y) , the value of a pel in a 
picture segment to be predicted is converted to a 
reference picture pel value (x', y') using the affine 
motion model as shown in the following equation. (2) . 
Under this assumption, all the affine parameters are 
searched and detected as affine motion parameters. 
Motion compensated prediction performed on each 
arbitrarily shaped prediction picture segment after the 
detection of affine motion parameters is proposed and 



described in "Motion Compensated Prediction Using An 
Affine Motion Model" (a technical report of IE94-36 by 
the Institute of Electronics, Information and 
Communication Engineers of Japan) . 

[y] =^(-s?ne ctse)(o\)(y) " ( t j 

(2) 

The definition of 6, (Cx, Cy) , (tx, ty) will be 
described later. 

Fig. 33 shows a concept of the motion compensated 
prediction process using the affine motion model. 

In the figure, 

i represents the position of a segment on a display 
used as a unit for motion compensated prediction; 

fi(x, y, t) represents a pel (x, y) in the segment 
position i and at time t; . 

Rv represents a translational displacement search 
range; 

Rrot, scale represents a search range for a rotated 
angle/scaled amount; 

V represents translational motion vector including 
translational motion parameters (=(tx, ty) ) ; 

rot is a rotation parameter (= a rotated angle 0) ; 

and 
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scale is scaled amount parameters ( =(Cx, Cy) ) • 
In the motion compensated prediction using the 
affine model, five af fine motion parameters including the 
rotated angle 6, scaled amount parameters (Cx, Cy) as 
well as the translational motion parameters (tx, ty) 
representing the motion vector must be detected- The. 
optimum affine motion parameters can be calculated by 
searching through all parameters- In order. to find the 
optimxam affine motion parameters, however, the nxomber of 
the arithmetic operations required is enormous- Thus, 
based on an assumption that the translational 
. displacement is predominant, two stages of affine motion 
parameter search algorithms are used- In the first 
stage, the translational displacement parameters (tx, ty) 
are searched. Then, in the second stage, the rotated 
angle 0 and the scaled amount parameters (Cx, Cy) are 
searched around the area which the translational 
displacement parameters (tx,ty) determined in the first 
stage represents. Further, minute adjustment for the 
translational displacement is implemented. Among the 
possible parameters, a combination of affine motion 
parameters representing the segment which has produced 
the minimum prediction error power is determined to be 
the combination of parameters for a prediction picture 
segment- Then, a difference between the prediction 
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picture segment and the current picture segment is 
calculated and regarded as a prediction error. And the 
prediction error is encoded. The prediction error power 
in accordance with motion compensated prediction using 
the affine motion model is given by the following 
equation (3) : 

Dv, rot. scale = {Mrot ' Mscale ' fi+v(x, y, t-l)-fi(x, Y, t) } ^ 

or 

= ElMrot • Mscale * fi+v(x, 7, t-l)-fi(x, y, t) | (3) 
y 

where. 



COS 8 slxiSj 
^ -sin8 cosSj 
^ c 6\ 



0 



Fig. 34 shows an example of a configuration of a 
conventional motion compensated predictor for performing 
motion compensated prediction using the affine motion 
model . 

In the figure, reference niomeral 220 is a 
translational displacement minute adjusted amount search 
range indication signal, reference numeral 221 is a 
rotated angle search range indication signal, 222 is a 
scaled amount search range indication signal, 223 is a 
translational displacement search range indication 
signal, 224 is a signal indicating a position of an input 
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picture segment on a display, and 225 is input picture 
segment data. Furthermore, reference numeral 226 is a 
horizontal displacement counter, 227 is a vertical 
displacement counter, 228 is a translational displacement 
adder, 229 is a first minimum prediction error power 
determinator, 230 is a memory readout address generator, 
231 is an interpolator, 232 is a half-pel interpolator, 
233 is a rotated angle counter, 234 is a scaled amount 
counter, 235 is a translational displacement/rotated 
angle/scaled amount adder, 236 is a second minim\im 
prediction error power determinator, 237 is a 
translational displacement minute adjusted amount 
counter, 238 is a translational displacement minute 
adjusted amount adder, and 239 is a final minimum 
prediction error power determinator . 

Fig. 35 is a flow chart showing the conventional 
operations of the above-mentioned motion compensated 
predictor. Fig. 36 is a flow chart showing the details 
of the affine motion parameters detection step 
illustrated at S224 of Fig. 35. 

In these flow charts, 

MV h[4] represents horizontal motion vector 
components (four possible components) ; 

MV v[4] represents vertical motion vector components 
(four possible components) ; 
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D_iain represents the minimum prediction error power; 
0 represents a rotated angle [radian] ; 
Cx and Cy represent scaled amount parameters; and 
tx and ty are motion vector minute adjusted amount 
parameters. 

Furthermore, D(e[i], Cx[i], Cy[i], tx[i], ty[i] ) 
represent the minimum prediction error power obtained 
after the detection of the affine motion parameters When 
MV_h[i] and MV_v[i] have been selected; 

de represents a rotated angle search pel point; 

dCx represents a horizontal scaled amount search pel 
point; 

dCy represents a vertical scaled amount search pel 
point; 

dtx represents a horizontal displacement minute 
adjusted amount search pel point; 

dty represents a vertical displacement minute 
adjusted amount search pel point; 

range_radian_juin represents a lower limit within a 
rotated angle search range; 

range_radian_max represents an upper limit within 
the rotated angle search range; 

range_scale__min represents a lower limit within a 
scaled amount search range; 

rangers cale__max represents an upper limit within the 
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scaled aino\ints search range; 

range_t_h_min represents a lower limit within a 
horizontal displacement minute adjusted amount search 
range ; 

range_t_h_max represents an upper limit within the 
horizontal displacement minute adjusted amount search 
range ; 

range_t_v min represents a lower limit within a 
vertical displacement minute adjusted amount search 
range ; 

range_t_v_max represents an upper limit within the 
vertical displacement minute adjusted amount search 
range ; 

D_min represents the minimum prediction error power 

(X, y) represents a position of a pel in an input 
picture segment to be predicted; 

f(x, y) represents the value of the pel (x, y) in 
the input picture to be predicted; 

fr(x, y) represents the value of a pel (x, y) in a 
reference picture; 

ax represents a value representing horizontal 
displacement obtained by using the affine motion model; 

ay represents a value representing vertical 
displacement obtained by using the affine motion model; 

D(ax, ay) represents a prediction error power 
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produced when ax arid ay are searched; and 

D(x, y) is a prediction error for the pel (x, y) 

when ax and ay are searched. 

Referring to Fig. 34 through Fig. 36, an operation 

of the conventional motion compensated prediction process 

using the affine motion model will be described in more 

detail. 

It is assumed in these figures that like elements or 
like steps which are given like reference niamerals and 
signs represent the same elements or represent the same 
processes. 
1) First Stage 

In the first stage of the conventional operation, 
detection of translational motion parameters {= the 
motion vector) obtained by the process similar to the 
above-mentioned block matching process is performed 
within a picture segment search range. 

Referring to Fig. 34, the picture segment search 
range is set through the horizontal displacement counter 
226 and the vertical displacement counter 227 by using 
the translational displacement search range indication 
signal 223. Then, the search pel points are moved. 
Through the translational displacement adder 228, the 
value indicating the position of a pel in an input 
picture segment is added to the counter values. Then, 



I 



10 



18 

the added result is supplied to the memory readout 
address generator 230/ and the pel value in a possible 
prediction picture portion is read out from the frame 
memory 219. The readout pel value is supplied to the 
pattern matching unit 213, and an error calculation 
operation similar to that used in the block matching 
method is performed- This matched result is supplied to 
the first minimum prediction error power determinator 229 
so as to obtain four possible translational motion 
parameters representing prediction errors in the reverse 
order of magnitude. These four possible translational 
motion parameters are expressed as MV_h[41 (horizontal 
components) and MV_v[4] (vertical components) - The 
operation of the first minimum prediction error power 
15 determinator 229 is similar to that of the minimum 

prediction error power determinator 216. These process 
steps correspond to steps S221 and S222 in Fig. 35. 
2) Second Stage 

2-1) Preparations (Picture Segment Search Range Setting 
. and Initialization of the Minimum Prediction Error Power) 
For each MV_h[i]/MV_v[i] (0sis3) , the rotated 
angle/ the scaled amount are searched around the minute 
space conterminous therewith. This operation corresponds 
to step S224 in Fig. 35 and the detailed process steps 
thereof are illustrated in Fig. 36, and will be described 
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in conjunction with the operation of the motion 
compensated predictor shown in Fig. 34. 

First, through the rotated angle search range 
indication signal 221 and the scaled amount search range 
indication signal 222, the rotated angle search range and 
the scaled amount search range are set in the rotated 
angle counter 233 and the scaled amount counter 234, 
respectively. Through the translational displacement 
minute adjusted amount search range indication signal 
220, the translational displacement search range is also 
set in the translational displacement minute adjusted 
amount counter 237. The second minimum prediction error 
power determinator 236 sets the value of the minimum 
prediction error power D_jiiin retained therein to MAXINT. 
These operations correspond to step S229 in Fig. 36. 
2-2) Rotated Angle Search 

The same operation is repeated for each of 
MV_h[i] /MV_v[i] (0^i^3) . Thus, a description about the 
rotated angle search will be directed to the case of 
MV_h[0] /MV_v[0] alone, and descriptions about other cases 
will be omitted. The affine motion models ax and ay- 
expressed in the following equations and obtained by 
changing the rotated angle 6 within the rotated angle 
search range while keeping the scaled amount parameters 
Cx and Cy and the translational motion minute adjusted 



amo\ints tx and ty unchanged: 

ax = dCx*cos(d0)*x + dCy*sin(de)*y + MV_h[i] + dtx 
ay = -dCx*sin(de)*x + dCy*cos (de) *y + MV_v[i] + dty (4) 
The absolute value for a difference between the pel 
value fr(ax, ay) in a reference picture segment and the 
pel value f (x, y) in an input picture segment is 
determined and accumulated to D(ax, ay) . 

Referring to Fig. 34, the above-mentioned operation 
is executed by fixing the counted values of the scaled 
amount counter 234 and the translational displacement 
minute adjusted amount counter 237, determining ax and ay 
given by equations (4) through the translational 
displacement/rotated angle/scaled amount adder 235 based 
on the counted value of the rotated angle counter 233, 
reading out the pels necessary for calculating fr(ax, ay) 
from the frame memory 219 through the memory readout 
address generator 230, calculating fr(ax, ay) from these 
pels through the interpolator 231, and determining the 
absolute value for the difference between the pel value 
f (X, y) in the input picture segment and the pel value 
fr(ax,. ay) in the reference picture segment through the 
pattern matching unit 213. Referring to Fig. 36, these 
operations correspond to steps S231 through S234. 

The above-mentioned operations are performed all 
around the rotated angle search range. Then, the rotated 



angle 6 which has produced the minimum prediction error 
within the rotated angle search range is determined 
through the second-stage minimum prediction error 
determinator 236. 
2-3) Scaled Amount Search 

The affine motion models ax and ay given by numeral 
equation (4) are also obtained by fixing the counted 
value of the translational displacement minute adjusted 
amount counter 237 as in the rotated angle search^ 
substituting the rotated angle 6 determined in 2-2) into 
numeral equation (4)^ and changing the scaled amount 
parameters Cx and Cy within the scaled amount search 
range . 

The scaled amount parameters Cx and Cy which have 
minimized D(ax, ay) are obtained by performing the 
operations similar to those in the rotated angle search. 
The scaled amount counter 234 counts scaled amount search 
pel points. 

2-4) Translational Displacement Minute Adjusted 
Amount Search 

The affine motion models ax and ay given by numeral 
equation (4) are also obtained by using the rotated angle 
e determined in 2-2) and the scaled amount parameters Cx 
and Cy determined in 2-3) and changing the value of the 
translational displacement minute adjusted amounts tx and 
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ty within the translational displacement minute adjusted 
cimount search range. 

Then, operations similar to those in the rotated 
angle search or the scaled amount search are performed. 
The translational displacement minute adjusted amount 
counter 237 counts translational displacement minute 
adjusted amount search pel points. In this case, tx and 
ty are searched with a half -pel precision. Then, the 
half -pel values for tx and ty are calculated through the 
half-pel interpolator 232, if necessary, before the half- 
pel value data for tx and ty are supplied to the pattern 
matching unit 213. The half-pel values are calculated as 
shown in Fig. 37 and as follows in the following equation 
(5), based on the spatial position relationship between 
half-pels and integer pels: 
t(x,y) = [l(xp.y,0 + Kxp+l.yp) + I(xp,yp+1) 

I(xp+l.yp+l)]/4; x,y:ODD 
[I(xp.yp) + I(xp+l,yp)]/2; x:ODD. y:EVEN 
[I(xp,yp) + I(xp.yp+l)]/2: x:EVEN. y:ODD (5) 
in which, both x and y are integers equal to or 
greater than zero. When x and y are both even niombers, 
half-pels having such coordinates of x and y will become 

integer pels. 

The process flow of the operations illustrated in 
Fig. 36 will be completed as described above. 



2-5) Final Affine Motion Parameters Determination 

A prediction error between a prediction picture 
segment and an input picture segment is then determined. 
This prediction error can be obtained by using e[i]/ 
Cx[i], Cy[i], tx[i], and ty[i] given by the above- 
mentioned affine motion parameters search from 2-2) 
through 2-4) for all of MV_h[i] and MV_v[i] . The picture 
segment position i and the set of affine motion 
parameters therefor which has given the smallest 
prediction error are regarded as the final search result. 
These operations correspond to steps S22S through S228 in 
Fig. 31. 

As described above, the affine motion parameters 
search requires an enormous calculational burden as well 
as a great many process steps. 

Fig. 38 is a diagram showing a method of calculating 
a non- integer pel value produced when the rotated angle 
and the scaled amount are searched. In other words, the 
figure is a diagram showing a method of calculating 
fr(ax, ay) through the interpolator 231. 

In the figure, O represents a real sample point in a 
picture, while • represents a virtual pel value obtained 
by performing the above-mentioned calculation method. 
fr(ax, ay) are represented by 1 (x, y) calculated in a 
reference picture and given by the following equation (6) 
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(in which x = ax, y = ay) : 
Kx. y) = WxiWyxI (xp, yp) + WxaWyil (xp+1, yp) + 

WxlWy2l (Xp, yp+1) + Wx2Wy2l (Xp+1, yp+1) 
Wx2 = X* - Xp 



(6) 



Wxl = 1. 0 - Wx2 

w,2 = y' - yp 

Wyl = 1. 0 - Wy2 

During the affine motion parameters search, pel 
matching is performed and the segment which has produced 
the minimum prediction error power is selected. 
Consequently, each time any of the above-mentioned five 
affine motion parameters is changed, the possible 
prediction picture segment should be formed again. In 
addition, rotation and scaling of an object produces non- 
. integer pel values. Thus, the operations expressed in. 
equation (6) are repeated over and over again during the 
affine motion parameters search. Consequently, the 
affine motion parameters search is very tedious and time- 
consiaming. 

As another motion compensation method using block 
matching for applying a simple enlarged or reduced 
picture, Japanese Unexamined Patent Publication No. HEI6- 
153185 discloses a motion compensator and an encoder 
utilizing the above motion compensation method. In this 
25 method, a reference picture portion included in a frame 



memory is reduced or enlarged by a thin-out circuit or 
interpolator^ and then a motion vector indicating the 
above reduction or enlargement is detected. In this 
configuration, a fixed block is extracted from the 
reference picture portion to perform an interpolation or 
a thin-out operation, instead of a complex arithmetic 
operation such as required by a motion compensation 
method using affine motion model. Namely, after 
implementing a predetermined process on an extracted 
fixed picture portion, the extracted picture* portion is 
compared with an input picture. The process is a simple 
and fixed one, so that this method can be applied only to 
a motion prediction of a picture such as simple reduction 
or enlargement of an input picture. 

The conventional motion compensated prediction 
methods are constituted and implemented as described 
above . 

In the first conventional motion compensated 
prediction iaethod using block matching, formation of a 
prediction picture portion is implemented by 
translational motion of a macroblock from a reference 
picture. Thus, the process itself is simple. However, 
in this process, only the translational displacement of 
an object can be predicted, and prediction performance 
deteriorates when rotation, scaling up and down, or 



zooming of the object are involved in the motion - 

On the other hand, in the second conventional motion 
compensated prediction method using the aff ine motion 
model, a prediction picture segment is formed using the 
aff ine motion model. Thus, when the motion of an object 
involves the more complicated types of motion such as 
rotation, this method can be applied. However, the 
operations needed for implementing the process according 
to this method are very complex and such a motion 
compensated predictor must be provided with a complex 
circuit having a large number of units. 

In general, as the motion compensated prediction 
process becomes more simplified, prediction often becomes 
less accurate. In contrast, the motion compensated 
prediction using the affine motion model increases 
prediction accuracy at the expense of more complex and 
tedious operations . 

Disclosure of the Invention 

The present invention has been made to solve the 
above-mentioned problems. It is, therefore, an object of 
the present invention to obtain an encoder using a 
relatively simple motion compensated prediction method 
capable of performing motion compensated prediction for 
various types of object motion which vary with time. In 
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order to achieve the above-mentioned objects, the real 
sample point pels on a reference picture, or the pels 
obtained by simple filtering, are used to form a 
prediction picture portion which having a different shape 
and size with respect to the picture portion to be 
predicted. 

According to the present invention, a video encoder 
for performing compression encoding of input video 
divided into predefined blocks, including a motion 
compensated prediction means for implementing a motion 
compensation by detecting a motion between frames of 
blocks of an input video, the video encoder comprises: 

a motion detector, including a transformed block 
matching unit for trans foirming a reference picture 
portion comprised of only integer pels, which consist of 
real sampling points, into a picture portion having a 
predefined shape, for determining addresses of pels of a 
transformed reference picture portion, and for matching 
the transformed reference picture portion with the block 
of input video comprised of integer pels, and the motion 
detector outputting a motion vector providing a minimum 
prediction error; and 

a motion compensator, including a corresponding pel 
determinator for transforming the reference picture 
portion based on a motion parameter obtained from a 



28 

comparison result of the transformed block matching unit, 
and for determining pels corresponding to pels of the 
reference picture portion, and outputting a prediction 

picture portion. 

Further, in the video encoder of the invention, the 
transformed block matching unit transforms a reference 
picture portion comprised of half-pels as well as integer 
pels into a picture portion having a predefined shape. 

The video encoder of the invention further comprises 
a preprocessor for separating an input video into video 
objects each of which is an encoding unit, the video 
encoder dividing the video object into blocks, detecting 
a motion, and compensating a motion of a divided video 
obj ect . 

Further, in the video encoder of. the invention, the 
transformed block matching unit transforms the reference 
picture portion comprised of half-pels as well as integer 
pels into one of a picture portion scaled down at a 
specific ratio and a picture portion scaled up at a 
specific ratio, determines the addresses of pels of a 
transformed reference picture portion, and matches the 
transformed reference picture portion with the input, 

picture portion; 

the corresponding pel determinator transforms the 
reference picture portion based on the motion parameter. 



determines addresses of pels of a transformed reference, 
picture portion, and outputs a motion prediction picture. 

Further, in the video encoder of the invention, the 
transformed block matching unit transforms the reference 
picture portion comprised of integer pels and half-pels 
into a picture portion rotated by a predefined angle, 
determines addresses of pels of a transformed reference 
picture portion, and matches the transformed reference 
picture portion with the input picture; and 

the corresponding pel determinator transforms the 
reference picture portion based on the motion parameter, 
determines addresses of pels of a transformed reference 
picture portion, and outputs a motion prediction picture 

In the video encoder of the invention, the 
transformed block matching unit transforms a reference 
picture portion comprised of integer pels and half -pels 
into a picture portion rotated by one of angles of ±45", 
±90°, ±135% and ±180'. 

Further, in the video encoder of the invention, the 
transformed block matching unit transforms the reference 
picture portion, which has been translated within a 
search range of the reference picture, into one of a 
picture portion scaled up, a picture portion scaled down 
a picture portion rotated by a predefined angle, 
determines addresses of pels of a transformed reference 
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picture portion, and matches the transformed reference 
picture portion with the input picture, and 

the corresponding pel determinator transforms the 
reference picture portion based on the motion parameter, 
determines addresses of pels of a transformed reference 
picture portion, and outputs a motion prediction picture. 

Further, in the video encoder of the invention, the 
transformed block matching unit has a transformation 
pattern table for transforming the reference picture 
portion, and matches the reference picture portion 
transformed according to the transformed value obtained 
from the transformation pattern table with the input 
picture comprised of integer pels and half-pels, and 

the corresponding pel determinator transforms the 
reference picture portion based on the motion parameter, 
determines addresses of pels of a transformed reference 
picture portion, and outputs a motion prediction picture. 

Further, in the video encoder of the invention, the 
transformed block matching unit selectively filters 
specific pels of the reference picture portion extracted 
for matching, and matches the transformed reference 
picture portion with the input picture portion. 

in the video encoder of the invention, the reference 
picture used for detecting the motion is the frame which 
is forward or backward in time and stored in a frame 
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memory for reference, and wherein the transformed block 
matching unit matches the transformed reference picture 
portion with the input picture using the frame. 

According to the present invention, a video decoder 
for decoding a compressed picture code of an input video 

data includes: 

an entropy decoder for receiving a motion vector 
extracting motion parameter from the input video data and 
representing a direction and- an amount of the motion and 
transformation pattern data showing transformation 

indication; and 

a motion compensator for transforming the integer 
pels of the reference picture portion corresponding to 
the frame into a predefined shape based on the motion 
parameter output from the entropy decoder according to 
the transformation pattern data included in the input 
video data and for generating a picture to be added to 
the input video data. 

In the video decoder of the invention, the motion 
compensator further computes pels of the corresponding 
reference picture portion comprised of half-pels as well 
as integer pels and transforms the corresponding 
reference picture portion into a picture portion having a 

predefined shape. 

According to the present invention, a video encoding 



method for performing compression encoding of an input 

digital video, comprises: 

a motion compensated prediction means for storing a 
reference picture, for dividing the reference picture 
into predefined blocks, and for detecting a motion 

between frames; 

transformed block matching step for transforming a 
reference picture portion comprised of integer pels into 
a picture portion having predefined shape, for 
determining addresses of pels of a transformed reference 
picture portion to generate a prediction picture portion, 
and for matching the transformed reference picture 
portion with the block of the input digital video; 

corresponding pel determining step for determining 
pels of the reference picture portion by using the 
address of transformed reference picture portion based on 
a motion vector providing a minimum error selected by the 
transformed block matching step, and for supplying a 
motion compensated output. 

In the video encoding method of the invention, the 
transformed block matching step transforms a reference 
picture portion comprised of half-pels as well as integer 
pels into a picture portion having a predefined shape as 
a reference, determines addresses of pels of a 
transformed reference picture portion to generate a 
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prediction picture portion, and matches the transformed 
reference picture portion with the input digital video. 

In the video encoding method of the invention, the 
transformed block matching step further comprises a 
transformation pattern table, transforms the reference 
picture portion based on transformation value 
corresponding to an address from the transformation 
pattern table, and matches the transformed reference 
picture portion with the input digital video- 

According to the present invention, a video decoding 
method for decoding a compressed picture code of an. input 
video data comprises: 

an entropy decoding step for receiving a motion 
vector extracting motion parameter from the input video 
data and representing a direction and an amount of the 
motion and transformation pattern data showing 
transformation indication; and 

a motion compensating step for transforming the 
reference picture comprised of integer pels corresponding 
to the frame into a predefined shape based on the motion 
parameter output from the entropy decoding step according 
to the transformation pattern data included in the input 
video data and for generating a picture to be added to 
the input video data. 

In the video decoding method of the invention, the 



motion compensating step transforms the reference picture 
portion comprised of half-pels as well as integer pels 
into a picture portion having a predefined shape. 
A video encoding/decoding system comprises: 
a video encoder for performing compression encoding 
of input video divided into predefined blocks, including 
a motion compensated prediction means for implementing a 
motion compensation by detecting a motion between frames 
of blocks of an input video, the video encoder 

comprising: 

a motion detector, including a transformed 
block matching unit for transforming a reference 
picture portion comprised of only integer pels, 
which consist of real sampling points, into a 
picture portion having a predefined shape, for 
determining addresses of pels of a transformed 
reference picture portion, and for matching the 
transformed reference picture portion with the block 
of input video comprised of integer pels, and the 
motion detector outputting a motion vector providing 
a minimum prediction error; and 

a motion compensator, including a corresponding 
pel determinator for transforming the reference 
picture portion based on a motion parameter obtained 
from a comparison result of the transformed block 
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matching unit, and for determining pels 
corresponding to pels of the reference picture 
portion, and outputting a prediction picture 
portion; and 

a video decoder for decoding an input compressed 
video data, the video decoder including motion 
compensated prediction means for implementing motion 
compensation by detecting a motion between frames, 

the motion compensated prediction means has a 
mechanism for determining pels of a corresponding 
picture portion comprised of predefined integer pels 
by transforming the corresponding reference picture 
portion into a predefined shape according to motion 
parameters included in the input compressed video 
data, for extracting the integer pels determined, 
and for adding the transformed reference picture 
portion and the decoded video signal so as to 
produce a decoded video data. 

Brief Description of the Drawings 

Fig. 1 is a block diagram showing a basic 
configuration of a video encoder according to the present 
invention. 

Fig, 2 is a block diagram showing an internal 
configuration of a motion detector 8 illustrated in Fig. 



Fig. 3 is a flow chart showing operations of the 
motion detector 8 configured as shown in Fig. 2. 

Fig. 4 is a conODination of explanatory drawings 
which describe an operation outline of a transformed 
block matching unit 21 according to a first embodiment 
the invent ion . 

Fig. 5 is a block diagram showing an internal 
configuration of the transformed block matching unit 21 

Fig. 6 is a flow chart showing operations of the 
transformed block matching unit 21. 

Fig. 7 is a block diagram showing an internal 
configuration of a motion compensator 9 illustrated in 
Fig. 1. 

Fig. 8 is a flow chart showing operations of the 
motion compensator 9. 

Fig. 9 is a explanatory drawing showing a picture 
object separating operation of a preprocessor 2. 

Fig. 10 is a block diagram showing an internal 
configuration of a motion detector 8b according to a 
second embodiment of the invention. 

Fig. 11 is a block diagram showing an internal 
configuration of a motion detector 8c according to a 
third embodiment of the invention. 

Fig. 12 is a combination of explanatory drawings 
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which describe an operation outline of a transformed 
block matching unit 42 • 

Fig. 13 is a block diagram showing an internal 
configuration of the transformed block matching unit 42. 

Fig. 14 is a flow chart showing operations of the 
transformed block matching unit 42. 

Fig. 15 is a combination of explanatory drawings 
which describe an operation outline of a transformed 
block matching unit 42b according to a fourth embodiment 
of the invention. 

Fig. 16 is a block diagram showing an internal 
configuration of the transformed block matching unit 42b. 

Fig. 17 is a flow chart showing operations of the 
transformed block matching unit 42b. 

Fig. 18 is a combination of explanatory drawings 
which describe other form of transformed block matching 
according to the fourth embodiment. 

Fig. 19 is a combination of explanatory drawings 
which describe other form of transformed block matching 
according to the fourth embodiment. 

Fig. 20 is a block diagram showing an internal 
configuration of a corresponding pel determinator 34 
according to a fifth embodiment of the invention. 

Fig. 21 is a combination of explanatory drawings 
which describe transformed block matching according to a 
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sixth embodiment of the invention. 

Fig. 22 is a explanatory drawing showing a filtering 
operation of integer pels constituting a prediction 
picture portion according to the sixth embodiment. 

Fig. 23 is a combination of explanatory drawings 
which describe an operation outline of a transformed 
block matching unit 42c. 

Fig. 24 is a block diagram showing an internal 
configuration of the transformed block matching unit 42c. 

Fig. 25 is a flow chart showing operations of the 
transformed block matching unit 42c. 

Fig. 26 is a block diagram showing an internal 
configuration of a motion compensator 9b according to the 
sixth embodiment. 

Fig. 27 is a flow chart showing operations of the 
motion compensator 9b according to the sixth embodiment. 

Fig. 28 is a block diagram showing a configuration 
of a video decoder according to a seventh embodiment of 
the invention. 

Fig. 29 is a combination of explanatory drawings 
which describe a concept of motion compensated prediction 
using block matching according to a first conventional 
related art. 

Fig. 30 is a block diagram showing a configuration 
of a motion compensated predictor (block matching 



section) of a video encoder according to the first 
conventional related art. 

Fig. 31 is a flow chart showing operations of the 
motion compensator according to the first conventional 
related art. 

Fig. 32 is a combination of explanatory drawings 
which describe a motion compensated prediction method 
used in the MPEGl video standard. 

Fig. 33 is a combination of explanatory drawings 
which describe a concept of motion compensated prediction 
using an affine motion model according to a second 
conventional related art. 

Fig. 34 is a block diagram showing a configuration 
of a motion compensator for performing motion compensated 
prediction using the affine motion model according to the 
second conventional related art. 

Fig. 35 is a flow chart showing operations of the 
motion compensated predictor according to the second 
conventional related art. 

Fig. 36 is a flow chart showing details of an affine 
motion pareimeters detection step illustrated in Fig. 35. 

Fig. 37 is a explanatory drawing showing half-pel 
interpolation implemented by a half-pel interpolater 232. 

Fig. 3 8 is a explanatory drawing which describes a 
non-integer pel interpolation method at a rotated 
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angle/ scaled amount search step executed by an 
interpolator 231 • 

Best Mode for Carrying Out the Invention 
Embodiment 1 . 

A video encoder and a video decoder according to the 
present invention may be used in^ for example, a digital 
video transmitting system/ a digital video recording 
apparatus, a digital video storage data base, or a 
digital video retrieval and reading system using a 
satellite, ground wave, or priority commxinication 
network. 

Fig. 1 shows a basic configuration of a video 
encoder. 

Referring to the figure, reference numeral 1 is an 
input digital video signal, reference numeral 2 is a 
preprocessor, reference numerals 3 and 13 are intra- 
/interframe (inside of the frame/between the frames) 
selectors, 4 is an orthogonal transformer, 5 is a 
quantizer, 6 is a dequantizer, 7 is an inverse orthogonal 
transformer, 8 is a motion detector, 9 is a motion 
compensator, 10 is a frame memory, 11 indicates motion 
parameters including the motion vector, 12 is prediction 
picture (image) data, 14 is an encoding controller, 15 is 
a mode selection prompting flag, 16 is an intra- 
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/interframe encoding indication flag, 17 is a quantizing 
step parameter, and reference numeral 18 is an entropy 
encoder, 19 is a compression video data. The essential 
elements of the present invention are the motion detector 
8 and the motion compensator 9. 

The operation of a video encoder according to the 
first embodiment of the present invention will now be 
described. 

The video encoder receives a video signal 1 
representing a frame which is a component of a color 
video sequence. The input video signal 1 is digitized, 
preprocessed in the preprocessor 2 where it is sxobjected 
to format conversion, and separated into block data. It 
is assumed herein that the separated block data includes 
a pair of the luminance signal component and color 
difference signal components spatially correspond 
thereto. From now on, the luminance signal component 
will be referred to as a luminance block, while the . color 
difference signal component will be referred to as a 
color difference block. 

Next, the intra-/interf rame encoding selector 3. 
determines whether each block data is subject to 
intraframe encoding or interframe encoding. When 
intraframe (inside of the frame) encoding has been 
selected, the block data representing input original 
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picture (image) data supplied from the preprocessor 2 is 
supplied to the orthogonal transformer 4. On the other 
hand/ when inter frame (between the frames) encoding has 
been selected, prediction error block data, which is a 
difference between the input original picture data 1 
supplied from the preprocessor 2 and the prediction 
picture data 12 supplied from the motion compensator 9, 
is supplied to the orthogonal transformer 4. This mode 
selection between the intraframe encoding and the 
interframe encoding may be implemented by the mode 
selection prompting flag 15 conditioned by the coitunand of 
the encoding controller 14. The selected encoding mode 
is transmitted to the entropy encoder 18 in the form of 
the intra-/interframe encoding indication flag 16 and 
multiplexed onto an encoded bitstream 19. 

The orthogonal transformer 4 uses an orthogonal 
transform such as Discrete Cosine Transform (DCT) - The 
orthogonal transforming coefficient is quantized by the 
quantizer 5 by using the quantizing step parameter 17 
prepared by the encoding controller 14. Then, the 
redundancy of the quantized orthogonal transforming 
coefficient is reduced through the entropy encoder 18 and 
multiplexed onto the encoded bitstream 19. At the same 
time, the quantized coefficient is dequantized by 
dequantizer 6 and is further subjected to the inverse 
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orthogonal transformation through the inverse orthogonal 
transformer 7 such that the prediction error signal is 
restored. A local, decoded picture is generated by 
adding the restored prediction error signal to the 
prediction picture data 12 supplied from the motion 
compensator 9. When the intra-/interf rame encoding 
indication flag 16 indicates the intraframe encoding 
mode, the zero signal will be selected via the encoding 
selector 13, and the prediction error signal will not be 
added to the prediction picture data. The local, decoded 
picture is used as a reference picture in motion 
compensated prediction for subsequent frames. 
Consequently, the local, decoded picture data are written 
into the frsime memory 10. 

Now, the motion compensated prediction method and 
apparatus, which is one of the important features of this 
embodiment, will be described. 

In this embodiment, it is assumed that block data 
separated by the preprocessor 2 is subjected to motion 
compensated prediction. The motion compensated 
prediction process is implemented by the motion detector 
8 and the motion compensator 9. Then, in the motion 
detector 8, the motion parameters 11 including the motion 
vector for the block subject to the motion compensated, 
prediction are detected. The motion compensator 9 uses . 



the motion parameters 11 and fetches the prediction 
picture data 12 from the frame memory 10. The motion 
detection process is implemented by using luminance 
blocks. The motion compensated prediction for color 
difference blocks uses the motion detected result of the 
luminance blocks. Now, a description will be directed to 
the motion compensated prediction for liaminance blocks 
alone . 

First, the motion detection process will be 
described. 

The motion detector 8 implements the motion 
detection process. The motion detector 8 searches within 
a predetermined reference picture range for a picture 
portion that is most similar to an input picture 
luminance block. Then, the parameters representing the 
change of the input picture block on the display are 
detected. In the conventional block matching method 
which has been described above in the conventional 
related art, a block which is most similar to an input 
picture liaminance block is searched. Then, the 
translational displacement of the input picture block on 
the display is detected in the form of the motion vector. 

The motion detector 8 according to this embodiment 
implements both conventional block matching using square 
blocks and block matching using transformed blocks which 
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will be described later. The motion detector 8 selects 
' data exhibiting the higher prediction accuracy obtained 
by using conventional block matching or transformed block 
matching. 

Now, the operation of the motion detector 8 
according to this embodiment will be described. 

Fig. 2 is a block diagram showing a detailed 
configuration of the motion detector 8 illustrated in 
Fig. 1. Fig. 3 is a flow chart showing the operations of 
the motion detector 8. 

Referring to Fig. 2, reference numeral 20 is a block 
matching unit, reference numeral 21 is a transformed 
block matching unit, 22 is a motion compensated 
prediction mode determinator, 23 is a motion vector 
obtained by implementing transformed block matching, 24 
is the minimiam prediction error produced by implementing 
transformed block matching, 25 is a final motion vector, 
and reference numeral 26 is a motion compensated 
prediction mode indication signal. It is assumed herein 
that the final motion vector 25 and the motion 
compensated prediction mode indication signal 26 are both 
included within the motion parameters 11. 

The internal configuration of the block matching 
unit 20 and the flow chart showing the operations of the 
block matching unit 20 are as illustrated in Figs. 43 and 
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3, D BM represents the minimum prediction error produced 
by implementing block matching, and D_DEF represents the 
minimum prediction error produced by implementing 
transformed block matching. 

Fig. 4 is a combination of explanatory drawings 
which describe an operational outline of the transformed 
block matching unit 21 which is one of the most essential 
structural elements of the present invention. Fig. 5 is 
a block diagram showing a detailed internal configuration 
of the transformed block matching unit 21. Fig. 6 is a 
flow chart showing the operations of the transformed 
block matching unit 21. 

Referring to Fig. 5, reference numeral 29 is a 
horizontal displacement search range indication signal, 
reference numeral 30 is a vertical displacement search 
range indication signal, 31 is a horizontal displacement 
counter, 32 is a vertical displacement counter, 33 is a 
rotated angle counter, 34 is a corresponding pel 
determinator, and reference numeral 35 is a memory 
readout address generator. The pattern matching unit 213 
and the minimum prediction error power determinator 216 
perform the same operations as those of the corresponding 
structural elements illustrated in Fig. 34. 

Referring to Fig. 6, 
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dx represents a horizontal displacement search pel 
point; 

dy represents a vertical displacement search pel 
point; 

range_h_min represents a lower limit within a 
horizontal search range; 

range_h_max represents an upper limit within the 
horizontal search range; 

range_v_iain represents a lower limit within a 
vertical search range; 

range_v_max represents an upper limit within the 
vertical search range; 

D_min represents the minimum prediction error power; 

D(dx, dy) is prediction error power produced when dx 
and dy are searched; 

(x, y) represents the position of a pel in an input 
picture block; 

(rx, ry) represents a pel in a reference picture 
corresponding to the pel (x, y) ; 

(rdX/ rdy) are rotated angle parameters; 
D(dX/ dy) represents a prediction error for the pel 
(x, y) when dx and dy are searched; 

f (x, y) represents the value of the pel (x, y) in 
the input picture; 

fr(x, y) represents the value of the pel (rx, ry) in 
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the reference picture; 

MV h represents a motion vector horizontal 

component ; 

MV_v represents a motion vector vertical component; 

ix represents an offset value for horizontal 
displacement (constant) ; 

iy represents an offset value for vertical 
displacement (constant) ; and 

block_size represents the size of the input picture 

block. 

1) Motion vector Detection By Using Block Matching 

The motion vector for the input picture block is 
determined through the block matching unit 20 by using 
the procedures and the operations described hereinbefore 
in relation to the conventional art. Consequently, the 
motion vector 217 and the minimum prediction error power 
D_BM 218 to be supplied from the block matching unit 20 
are obtained. These operations correspond to step SI in 
Fig. 3. 

2) Motion Vector Detection By Using Transformed Block 
Matching 

Then, the transformed block matching process is 
implemented through the transformed block matching unit 
21 (step S2 in Fig. 3). 

Now, the transformed block matching process will be 



49 

described in more detail. It is assiimed herein that an 
input picture block which includes 8 x 8 integer pels is 
used as a unit for implementing the transformed block 
matching process. 
2-1) Operation Outline 

Fig. 4 shows an outline of the transformed block 
matching process implemented by the transformed block 
matching unit 21. 

Referring to the figure, an input picture 27 is 
encoded by using motion compensated prediction. For 
example, a frame (picture) within the preprocessor 2, 
that is, a reference picture 28 is a local decoded frame 
(picture) which had been encoded prior to the input 
picture 27 and stored in the frame memory 10, O 
represents a real sample integer pel in a frame portion 
represented by the luminance signal, X represents a half- 
pel interposed midway between the real sample integer 
pels. Herein, (the luminance block of) the input picture 
block is a picture portion including 8x8 (integer pels) 
in the input picture 27, while the transformed block is a 
picture portion including □ pels in the reference picture 
28 of possible prediction picture. The output of the 
frame memory 10 and the output of the preprocessor 2, 
both of which are indicated by parenthesized reference 
numerals in Figs. 1 and 2, are extracted and supplied to 
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the transformed block matching unit 21 in the motion 
detector 8 and compared. 

In this embodiment^ a transformed block is defined 
to be a picture portion rotated 45 degrees clockwise or 
counterclockwise relative to the luminance block and 
having four sides /2 times scaled up from the sides of 
the input picture block. That is, a length of the 
reference picture block becomes 17/2" times. Intervals of 
the pels within the reference picture block becomes 
identical to horizontal/vertical length of intervals of 
the sample points of the input digital picture 1 of the 
frame. The transformed block in this embodiment includes 
only integer pels from the reference picture 28. Namely, 
transformed block matching according to this embodiment 
is a process for finding, within a given search range, a 
transformed block that is most similar to the luminance 
block of the input picture block including 8x8 integer 
pels in the reference picture 28 as shown in Fig. 4. 
2-2) Initial Settings (Transfonaed Block Search Range 
Setting, Initial Value Setting) 

Because an input picture block has a different shape 
than a possible prediction picture portion, it is 
necessary to specify the starting point of the motion 
vector to be detected. Namely, a one-to-one 
correspondence is established in advance between the 
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integer pels constituting the input picture luminance 
block and the integer pels constituting a transformed 
block within a prediction picture portion. 

Herein, as shown in the dotted arrows in Fig. A, the 
integer pel at the upper left corner of an input picture 
block is made to correspond to the integer pel at the 
left top of a transformed block. In other words, the 
possible prediction picture portion is a picture portion 
rotated 45. degrees clockwise relative to the transformed 
block within the reference picture 28 and having four 
sides l/yf2 scaled down with respect to the sides of the 
transformed block. A change in this correspondence will 
lead to a change in the rotated <iirection of the possible 
prediction picture portion. In addition, because this 
one-to-one correspondence is also established between the 
integer pels of the input picture block and the integer 
pels of the possible prediction picture portion, motion 
detection similar to that implemented by using block 
matching can be performed. 

Namely, a shape for extracting picture portion of 
the reference picture 28 in Fig. 4 is patternized for 
block matching and indicated by a predetermined address 
(pel position) comprised of only integer pels. The 
difference between the pel of the address and the pel 
included in the input picture 27 corresponding to the 
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pels of an original picture data is accumulated to 
determine a minimum prediction error power. 
Consequently, block matching is implemented only by 
indicating . the address without complex operation, which 
enables a high-speed block matching. Furthermore, by 
providing a various addressing (indicating pel position), 
an indication can be flexible to extract the reference 
picture portion such as rotating, combination of rotating 
and enlarging or reducing other than simply enlarging, 
reducing . 

More specifically, a transformed block search range 
for implementing transformed block matching is set in. the 
horizontal displacement counter 31 and the vertical 
displacement counter 32 through the horizontal 
displacement search range indication signal 29 and the 
vertical displacement search range indication signal 30. 
Then, in the minimum prediction error power determinator 
216, the minimum prediction error power D_jain is set to a 
maximum integer value MAXINT (OxFFFFFFFF, for example) . 
These operations correspond to step S4 in Fig. 6. 
2-3) Block Transformation Parameters Setting 

In this embodiment, rdx and rdy shown in steps S6 
and S8 in Fig. 6 will be used as block transformation 
parameters. The setting of these parameters is 
implemented by the rotated angle counter 33 such that the 
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block to be transformed within the reference picture 
illustrated in Fig. 4 is rotated 45 degrees clockwise. 
The value of y is regarded as the initial value of rdx or 
rdy. Then, each time x is incremented, rdx is 
incremented, and rdy is decremented. These operations 
correspond to steps S6 through S8 in Fig. 6. The setting 
in this way is implemented such that the block to be 
transformed is rotated clockwise. If the setting is done 
as rdy = -y at step S6, and ry = iy+(rdy++) at step S8, 
the block to be transformed will be rotated 
coianterclockwise. The operation of step S8 can be also 
represented as rx = ix+(rdx+l) and ry = iy+(rdy-l) . 

Namely, an addressing of pels to be extracted from 
the reference picture 28 is indicated at step S8. An 
address of integer pels rotated 45 degrees clockwise is 
indicated as next pels rx, ry. At step S12, this 
operation is repeated until addressing of the pel equals 
to block size of x, and is also repeated until addressing 
of the pel equals to block size of y at step S14. In 
this way, a prediction error of the pel extracted by 
addressing at step S8 is detected at step S9, and the 
error is accumulated at step SIO. As shown in the 
operation flow of Fig. 6, no computation is needed, and 
high-speed processing can be performed. The operation of 
step SIO can be also represented as D(dx,dy) = 
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D(cix,dY) +D (X, y) . Similarly^ the operations of steps Sll, 
S13, S17 and S19 are represented as x = x+1, y = y+1- 
This can be also said in flowcharts which will be 
described hereinafter, 

2-4) Possible Prediction picture Portion Readout 
Operation 

First, the reference picture portion pel (rx, ry) 
corresponding to the input picture block pel (x, y) in 
the input picture luminance block is determined. In 
other words, the initial positioning correspondence 
between, the integer pels of the input picture block and 
the integer pels of the possible prediction picture 
portion shown in Fig. 4 is established. The 
corresponding pel determinator 34 implements this 
operation. As shown at step S8 in Fig. 6, rx and ry can 
be obtained by adding rdx and rdy obtained in 2-3) to the 
predetermined offset value ix and iy, respectively. 
Then, data on a reference picture portion pel (rx+dx, 
ry+dy) is fetched from the frame memory. The memory 
readout address generator 35, shown in Fig. 5, receives 
the value of dx from the horizontal displacement counter 
31, the value of dy from the vertical displacement 
counter 32, and the value of rx and ry from the 
corresponding pel determinator 34 so as to generate the 
address for the pel (rx+dx, ry+dy) in the frame memory. 
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2-5) Prediction Error Power Calculation 

First/ the prediction error power D{dX/ dy) , 
produced from the motion vector indicating the pel 
position (dx, dy) , is initialized to zero. This 
operation corresponds to step S5 in Fig. 6* The absolute 
value for a difference between the pel value read out in 
2-4) and the value of the pel corresponding thereto in 
the input picture luminance block is accumulated to D(dX/ 
dy) • This operation is repeated until the condition x = 
y = block_size holds (in this case, the operation is 
repeated until block_size = 8 holds) . Then^ the 
prediction error power D{dx, dy) produced from the motion 
vector indicating the pel position (dx, dy) can be 
obtained. The pattern matching unit 213 illustrated in 
Fig. 5 implements this operation. The pattern matching 
unit 213 supplies D(dx, dy) to the minimum prediction 
error power de terminator 216 through the prediction error 
power signal 215. 

These operations correspond to steps S9 through SI 4 
in Fig. 6. 

2-6) Minimum Prediction Error Power Updating 

It is then determined whether D(dx, dy) obtained in 
3) has produced the minimum prediction error power among 
all the searched results that have been obtained so far. 
The minimum prediction error power determinator 216 



illustrated in Fig. 5 makes this determination. This 
operation corresponds to step S15 in Fig. 6. The minimum 
prediction error power determinator 216 compares the 
value of the minimum prediction error power D_min 
retained therein with the value of D(dx, dy) received 
through the prediction error power signal 215. If only 
the value of D(dx, dy) is smaller than D_min, D__min is. 
updated to the value of D(dx, dy) . Further, the minimum 
prediction error power determinator 216 retains the value 
of (dx, dy) at that time as the possible motion vector 
(MV_h, MV_v) • This updating operation corresponds to 
step S16 in Fig. 6. 

2-7) Motion Vector Value Determination 

The above-mentioned operations from 2-2) through 2- 

6) are repeated for all the search pel points (dx, dy) 
within the transformed block search range (steps S17 

through S20 in Fig. 6) . The finally retained (MV^h, 
MV_v) within the minimum prediction error power 
determinator 216 are output as the motion vector 23. 

As described above, a prediction picture portion 
which has produced the minimum prediction error power and 
therefore is most similar to an input picture block is 
searched. The displacement from the predetermined 
starting pel point of the selected prediction picture 
portion as a result of the transformed block search is 
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obtained and expressed in terms of the motion vector 23. 
The prediction error power D_DEF 24 produced at that time 
is also retained, 

3) Final Motion Compensated Prediction Mode Determination 

Next, the minimum prediction error power D_BM 218 
supplied from the block matching unit 20 and the minimum 
prediction error power D_DEF 24 supplied from the 
transformed block matching unit 21 are compared by the 
motion compensated prediction mode determinator 22. The 
smaller of the prediction error powers is selected to 
determine the final motion compensated mode between block 
matching and transformed block matching. This operation 
corresponds to step S3 in Fig. 3. 

The motion compensated prediction mode determinator 
22 supplies the final, selected motion compensated 
prediction mode indication signal 26 and the final motion 
vector 25 to the motion compensator 9 and the entropy 
encoder 18 in the form of motion parameters 11. 

Next/ the motion compensation process will be 
described. 

The motion compensator 9 implements the motion 
compensation process. The motion compensator 9 extracts 
prediction picture portion data from reference picture 
data according to the motion parameters 11 supplied from 
the motion detector 8. The motion compensator 9 
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according to this embodiment supports both the 
conventional square-block-based block matching and 
transformed block matching using a specific transformed 
block as disclosed herein. According to the motion 
compensated prediction mode represented by the motion 
parameters 11, the motion compensator 9 switches the 
operation between the conventional block matching and the 
transformed block matching to provide optimal 
performance. 

Now, the operation of the motion compensator 9 
according to this embodiment will be described. 

Fig. 7 is a block diagram showing a configuration of 
the motion compensator 9 of Fig. 1. Fig. 8 is a flow, 
chart showing the operations of the motion compensator 9. 

Referring to Fig. 1, reference numeral 37 is a 
corresponding pel determinator and reference numeral 38 
is a memory readout address generator. 
1) Corresponding Pel Determination 

By using an input picture block position indication 
signal 206 and the motion parameters 11 supplied from the 
motion detector 8, the sample points of the input picture 
block corresponding to those of the prediction picture 
portion in the reference picture 28 are determined. This 
operation corresponds to step S21 in Fig. 8. The 
corresponding pel determinator 37 illustrated in Fig. 7 
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implements this process. When the motion compensated 
prediction mode^ represented by the motion paraoueters 11, 
indicates block matching, the corresponding pels will 
become the sample points of a reference picture 
translated from the position indicated by the input 
picture block position indication signal 206 to the area 
specified by the motion vector. This operation 
corresponds to step S204 in Fig. 31, and is an operation 
for determining the position of the pel (x+dx, y+dy) 
within the reference picture 28 when dx and dy are 
expressed in terms of the motion vector. When the motion 
compensated prediction mode represented by the motion 
parameters 11 indicates transformed block matching, the 
corresponding pels will become the sample points of a 
reference picture block rotated by a specified angle 
indicated by the input picture block position indication 
signal 206, as described in the explanation of the motion 
detector 8 in the above 2-4) , and then translated to the 
area specified by the motion vector. This operation 
corresponds to step S9 in Fig. 6, and is an operation for 
determining the position of the pel (rx+dx, ry+dy) within 
the reference picture 28 when dx and dy are expressed in 
terms of the motion vector. 

2) Prediction picture Data Readout Operation 

The following operations correspond to steps S22 
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through S25 in Fig. 8. The memory readout address 
generator 38 receives the output of the corresponding pel 
determinatbr 37 and generates the memory address 
specifying the position of the prediction picture portion 
within the reference picture 28 to be stored in the frame 
memory 10, 

When the prediction picture portion includes half- 
pels, half -pel values are interpolated by the half -pel 
interpolator 232. This operation is implemented before 
the prediction picture data is supplied from the motion 
compensator 9, and corresponds to steps S23 and S24 in 
Fig- 8. Whether or not the prediction picture portion 
includes half-pels is determined by the corresponding pel 
de terminator 37 according to the motion vector value 
included in the motion parameters 11- Then, the 
determined result is supplied to a selective switch 36. 

The transformed block matching unit 21 configured as 
illustrated in Fig. 5 generates the corresponding pel 
points for a prediction picture portion that includes 
only real sample points as shown in Fig. 4. However, 
when the prediction picture portion includes half-pels as 
well, the transformed block matching unit 42 in Fig. 13 
will utilize configurations including the half-pel 
interpolator 232 as will be. described later. 

After aforementioned processes, the final prediction 
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picture data 12 is supplied. In this embodiiaent/ a 
.description is directed to the case where a transformed 
block has been formed by rotating an input picture block 
by 45 degrees. The rotated angle may be specified 
arbitrarily, such as 90 degrees, 135 degrees, 180 
degrees, and the like. In addition, depending on the 
determination of the values of dx and dy, other forms of 
rotation can be implemented - 

Further, in this embodiment, a frame-based video 
encoder is described. When an input digital video 
sequence is separated into a plurality of picture objects 
through the preprocessor 2 and each of the picture 
objects (a picture portion haying a common feature such 
as a motion feature, a pattern feature, etc. or a camera 
subject, and the like) is regarded as or is included in a 
block-based picture portion, the present invention may be 
applied to this object-based video encoder. 

Object-based encoding, for example, may be 
implemented by regarding a person with a still background 
as a single picture object as shown in Fig. 9, dividing 
the portion surrounding the person into a plurality of 
subblocks, and encoding data on the subblocks including 
the person as effective block data. Processes similar to 
those required for performing the above-described 
transformed block matching and motion compensation may 
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also be applied to this case. This understanding also 
applies to other embodiments which will be described 
hereinafter. 

In this embodiment, an encoder using an orthogonal 
transform encoding method is described. It is to be 
understood that the present invention may be applied to 
encoders using other encoding methods for encoding a 
motion compensated prediction error. This understanding 
also applies to other embodiments which will be described 
hereinafter. 
Embodiment 2 • 

Approximate displacement of a picture portion which 
will be subject to the transformed block matching process, 
is determined by the value of the motion vector 
representing the translational motion of an object. 
Consequently, when the picture portion which will be 
subjected to the transformed block matching process is 
defined to be the portion represented by the motion 
vector 217 supplied from the block matching unit 20 and 
when that picture portion is transformed and compared, 
the number of processing steps and the amount of 
processing time can be reduced. In this embodiment, a 
configuration which allows the above-mentioned 
determination of approximate displacement will be 
described. This method can be applied to other 
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embodiments which will be described hereinafter. 

This embodiment will describe another configuration 
for motion detector 8, 

Fig, 10 is a block diagram showing an internal 
configuration of the motion detector 8b according to this 
embodiment. Reference numeral 39 is a transformed block 
matching unit, reference nxjiaeral 40 is an adder, and 
reference numeral 41 is an initial search pel position 
indication signal. The transformed block matching unit 
39 uses the initial search pel position indication signal 
41 instead of the input 206. Other operations of the 
transformed block matching xinit are the same as those of 
the transformed block matching unit 21 described in the 
first embodiment. 

Fig. 10 shows a concrete circuit of the motion 
detector for obtaining approximate values . 

Referring to Fig. 10, the motion vector 217 supplied 
from the block matching unit 20 is added to the input 
picture block data 205 by. the adder 40. Instead of the 
input picture block position indication signal, the added 
result of the adder 40 is supplied to the transformed 
block matching unit 39 as the initial search pel position 
indication signal 41. The transformed block search range 
set through the horizontal displacement search range 
indication signal 29 and the vertical displacement search 
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range indication signal 30 is set so as to be smaller 
than those set in the first embodiment, Thus^ the time 
required for implementing the repetitive operations from 
steps S17 through S20 in Fig, 6 can be reduced. 
Embodiment 3. 

In the first and second embodiments, the transformed 
block portion in the reference picture 28 includes only 
integer pels. In this embodiment, the transformed block 
in the reference picture 28 includes half-pels as well as 
integer pels. 

This embodiment is different from the first 
embodiment in that the motion detector 8c and the motion 
compensator 9 illustrated in Fig. 1 have different 
internal configurations, respectively. The operation of 
the transformed block matching unit in a motion detector 
and the operation of the corresponding pel determinator 
of a motion compensator according to this embodiment are 
different from those in the first embodiment. The 
operations of other elements are the same as those in the 
first embodiment. Consequently, the following 
description will be directed to the operation of the 
transformed block matching unit and the operation of the 
motion compensator. As in the first embodiment, the 
operation of the motion detector 8c and the operation of 
the motion compensator 9 will be described separately. 
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Fig. 11 is a block diagram showing an internal 
configuration of the motion detector 8c according to this 
embodiment. Fig. 12 is a combination of explanatory 
drawings which describe an operation outline of the 
transformed block matching unit 42 which is one of 
important features of the present invention. Fig. 13 is 
a block diagram showing a detailed internal configuration 
of the transformed block matching unit 42. Fig. 14 is a 
flow chart showing the operations of the transformed 
block matching unit 42. 

Referring to these figures^ like structural elements 
and like steps which are given like reference niomerals 
and signs represent the same elements or perform the same 
processes . 

First, the operation of the transformed block 
matching unit 42 will be described. 
1) Operation Outline 

Fig. 12 is a combination of explanatory drawings 
which describe an operation outline of the transformed 
block matching unit 42. 

Referring to the figure, as in the first embodiment, 
reference numerals 27 and 28 refer to an input picture to 
be predicted and a reference picture, respectively. 
Reference sign O means a real sample point (integer pel) 
in a frame represented by luminance signals, and 
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reference sign X means an interposed pel (half-pel) 
interposed midway between the integer pels. It is 
assumed herein that the input picture 27 including 8x8 
(integer pels) is (a luminance block of) an input picture 
block* A picture portion comprised of □ pels within the 
reference picture 28 is a transformed block in a possible 
prediction picture portion herein. 

In this embodiment, a transformed block is defined 
to be a picture portion rotated 45 degrees clockwise or 
counterclockwise relative to the luminance block and 
having four sides l/y/2 times scaled down from the sides 
of the input picture block. That is, a length of the 
reference picture block becomes y/2 times of the length of 
the input picture. Intervals of the pels within the 
reference picture block becomes identical to 
horizontal/vertical length of intervals of the sample 
points of the input digital picture 1 of the frame. This 
block includes half-pels in addition to integer pels . 
within the reference picture 28. Transformed block 
matching according to this embodiment is a process for 
finding, within the reference picture 28, a transformed 
block portion which is most similar to the input picture 
luminance block including 8x8 samples (hereinafter, 
samples mean integer pels or half-pels) as illustrated in 
Fig. 12. 
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2) Initial Settings (Transformed Block Search Range and 
Initial Value Setting) 

As in the first embodiment, a one-to-one 
correspondence is established in advance between the 
integer or half-pels constituting the input picture 
l\aminance block and the integer or half-pels constituting 
a transformed block within a possible prediction picture 
portion. Herein, as shown in the dotted arrows in Fig. 
12, the pel at the upper left corner of the input picture 
block is made to correspond to the pel at the left top of 
the transformed block. Referring to the figure, the pel 
at the left top of the transformed block is a half-pel 
which indicates that the motion vector represents a 
picture portion comprised of half-pels in addition to 
integer pels. A possible prediction picture portion 
herein means a picture portion rotated 45 degrees 
clockwise relative to the transformed block within the 
reference picture 28 and having four sides /2 times 
scaled up with respect to the sides of the transformed 
block in the reference picture. A change in this 
correspondence will lead to a change in the rotated 
direction of the possible prediction picture portion. In 
addition, establishment of this one-to-one correspondence 
between the pels will lead to other one-to-one 
correspondences between the pels of the transformed block 
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and the pels of the input picture block. This one-to-one 
correspondence allows motion detection similar to that 
implemented by. using block matching. The operations 
needed for setting the transformed block search range by 
using the transformed block matching unit are the same as 
those described in the first embodiment- The structural 
elements needed for performing transformed block search 
range setting are illustrated in Fig. 13. This operation 
corresponds to step S26 in Fig. 14. 

3) Block Transformation Parameters Setting 

As in the first embodiment, rdx and rdy shown in 
Fig. 14 will be used as block transformation parameters. 
The setting of these parameters is implemented by the 
rotated angle counter 45. The value of y is regarded as 
the initial value of rdx or rdy. Then, each timex is 
incremented, rdx is incremented by 0.5, and rdy is 
decremented by 0.5. These operations correspond to steps 
S28 through S30 in Fig. 14. The setting of the 
parameters rdx and rdy in this way is implemented such 
that the block to be transformed is rotated clockwise. 
If the setting is performed as rdy = -y at step S28 and 
ry = iy+ (rdy+=0.5) at step S30, the block to be 
transformed will be rotated counterclockwise. 

4) Possible Prediction picture Portion Readout Operation 

First, the reference picture portion pel (rx, ry) 
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corresponding to the input picture block pel (x, y) in 
the input picture luminancia block is determined. The 
corresponding pel determinator 46 implements this 
operation. As shown in step S30 in Fig. 14, rx and ry 
can be obtained by adding rdx and rdy obtained in 3) to 
the predetermined offset values ix and iy, respectively. 
Then, data on the reference picture portion pel (rx+dx, 
ry+dy) is fetched from the frame memory. 

Next, pel (rx+dx, ry+dy) is fetched from the frame 
memory. The memory readout address generator 47 
illustrated in Fig. 13 receives the value of dx from the 
horizontal displacement counter 31, the value of dy from 
the vertical displacement counter 32, and the value of rx 
and ry from the corresponding pel determinator 46 so as 
to generate the address for the pel (rx+dx, ry+dy) to be 
stored in the frame memory. Readout data is used to 
interpolate the value of a half-pel through the half-pel 
interpolator 232, if necessary, as shown in step S31 in 
Fig, 14. . 

5) Prediction Error Power Calculation 

First, the prediction error power D(dx, dy) produced 
when dx and dy are expressed in terms of the motion 
vector is initialized to zero. This operation 
corresponds to step S27 in Fig. 14. The absolute value 
for a difference between the pel value read out in 4) and 
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the corresponding pel value of the input picture 
luminance block is accxamulated to D(dx, dy) . This 
operation is repeated until the condition x = y = 
block_size (in this case, block_size = 8) holds. Then, 
the prediction error power D(dx, dy) produced when the 
motion vector indicates the pel position (dx, dy) can be 
obtained. The pattern matching unit 213 illustrated in 
Fig. 13 implements this operation. The pattern matching 
unit 213 supplies D(dx, dy) to the minimum prediction 
error power determinator 216 using the prediction error 
power signal 215. These operations correspond to steps 
S32 through S37 in Fig. 14. 

6) Minimum Prediction Error Power Updating 

It is determined whether D{dx, dy) obtained in 5) 
has produced the minimxim prediction error power among all 
the searched results that have been obtained so far. The 
minimvim prediction error power determinator 216 
illustrated in Fig. 13 makes this determination. This 
determination process . corresponds to step S38 in Fig. 14. 
The determination process is the same as that in the 
first embodiment. The value of (dx, dy) at that time is 
retained as the possible motion vector (MV_h, MV_v) . 
This updating process corresponds to step S39 in Fig- 14. 

7) Motion Vector Value Determination 

The above-mentioned operations from 2) through 6) 
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are repeated f or . all the search pel points (dx, dy) 
within the transformed block search range (these 
operations correspond to steps S40 through S43 in Fig. 
14) • The finally retained value (MV_h, MV_v) within the 
minimum prediction error power determinator 216 are 
. output as the motion vector 43 . 

As described above, a prediction picture portion 
which has produced the minimum prediction error power 
and/ therefore, is most similar to an input picture block 
is searched by using transformed block matching. As the 
result of the prediction picture portion search, the 
displacement from the predetermined starting pel point of 
the selected prediction picture portion is obtained and 
expressed in terms of the motion vector 43 . The 
prediction error power D_DEF 44 at that time is also 
retained. 

The above-mentioned motion vector 43 and the 
prediction error power D_DEF 44 are used to determine the 
final motion compensated mode. The final motion 
compensated mode is thus determined. This determination 
method is the same as that in the first embodiment. 

Next, the motion compensation process will be 
described. 

The motion compensator 9 implements the motion 
compensation process. In this embodiment, the operation 
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of the corresponding pel de terminator 37 is different 
from that in the first embodiment. Consequently, the 
following description will be directed only to the 
operation of the corresponding pel determinator 37. The 
overall motion compensation operational flow is 
illustrated in the form of the flow chart in Fig. 8. 

In this embodiment, the corresponding pel 
determination is performed as follows: 

When the motion compensated prediction mode 
represented by the motion parameters 11 indicates block 
matching/ the corresponding pels will become the san^le 
points of a reference picture portion translated from the 
position indicated by the input picture block position 
indication signal 206 to the area specified by the motion 
vector- This process corresponds to step S204 in Fig. 
31, and represents an operation for determining the 
position of the pel (x+dx, y+dy) within the reference 
picture 28 when dx and dy are expressed in terms of the 
motion vector. 

When the motion compensated prediction mode 
represented by motion parameters 11 indicates transformed 
block matching, the corresponding pels are expressed with 
the sample points of a reference picture portion rotated 
by a specified angle indicated by the input picture block 
position indication signal 206 and then translated to the 
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area specified by the motion vector as described in 4) of 
the explanation of the motion detector 8. This operation 
corresponds to step S32 in Fig. 14, and is an operation 
for determining the position of the pel (rx+dx, ry+dy) 
within the reference picture 28 when dx and dy are 
expressed in terms of the motion vector • 

The prediction picture data readout operation and 
prediction picture generation are the same as those 
described in the first embodiment* 
Embodiment 4* 

In this embodiment/ a description will be directed 
to the case where a transformed block is a reduced input 
picture block. With regard to the case where a 
transformed block is enlarged relative to an input 
picture block, the operation will be similar and the 
description will be omitted. Thus, simplified 
trans foriaed block matching and motion compensation will 
be described. 

Now, the operation of the transformed block matching 
unit 42b in the motion detector and the operation of the 
corresponding pel determinator in the motion compensator 
will be described in detail with reference to Fig. 16 ^ 
For simplifying the description, it is .assumed herein 
that the transformed block matching unit 42b is a 
variation of the transformed block matching unit 42 



illustrated in Fig. 13. The transformed block matching 
unit 42b receives the same inputs as those with the 
transformed block matching unit 42, and supplies 
variations of the motion vector 43 and the prediction 
error power 44. The corresponding pel determinator in 
the motion compensator 9 is also a variation of the 
corresponding pel determinator 37 illustrated in Fig. 7. 
Consequently, this embodiment uses the transformed block 
matching unit 42b and the corresponding pel determinator 
37. 

Fig. 15 is a combination of explanatory drawings 
showing the operation outline of the transformed block 
matching unit 42b according to this embodiment. Fig. 16 
is a block diagram showing a detailed internal 
configuration of the transformed block matching unit 42b 
Fig. 17 is a flow chart showing the operations of the 
transformed block loatching unit 42b. 

In these figures, the elements and steps assigned 
the same reference numerals with the above-mentioned 
drawings will mean the same elements and operations. 

First, the operation of the transformed block 
matching unit 42b will be described. 
1) Operation Outline 

Fig. 15 shows the operation outline of the 
transformed block matching unit 42b. An input picture 
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27, a reference picture 28 and signs used within the 
pictures are the same as described above. In this 
embodiment, a transformed block is defined to be a 
picture portion having four sides which are half as large 
as those of an input picture luminance block. 
Transformed block matching according to this embodiment 
is a process for finding, within the given search range, 
a transformed reference picture block portion most 
similar to the input picture luminance block consisting 
of 8 X 8 samples as illustrated in Fig. 15. 
2) Initial Settings (Transformed Block Search Range 
Setting and Initial Value Setting) 

As in the first embodiment, a one-to-one 
correspondence is established in advance between the pels 
constituting an input picture liaminance block and the 
pels constituting a transformed block within a possible 
prediction picture portion. Herein, as shown in the 
dotted arrows in Fig. 15, the pel at the upper left 
corner of the input picture block is made to correspond 
to the pel at the left top of the transformed block. 
Because a one-to-one correspondence is established 
between the pels of the input picture block and the pels 
of the possible prediction picture portion, motion 
detection similar to that implemented by block matching 
can be performed. The operation needed for setting the 
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transformed block search range by using the transformed 
block matching unit is the same as that in the first 
embodiment. Fig. 16 illustrates the structural elements 
needed for performing transformed block search range 
setting. This operation corresponds to step S44 in Fig 
17. 

3) Possible Prediction picture Portion Readout Operation 

In this embodiment, specific block transformation 
parameters will not be used. As shown in step S47 in 
Fig. 17, the reference picture portion pel (sx, sy) 
corresponding to the input picture liaminance block pel 

(X, y) is obtained by adding x/2 and y/2 to the 
horizontal offset constant ix and the vertical offset 
constant iy, respectively. This corresponding pel 
determination is made by the corresponding pel 
detenninator 48. Then, the reference picture portion pel 

(sx+dx, sy+dy) is fetched from the frame memory 10. The 
memory readout address generator 49. illustrated in Fig. 
16 receives the value of dx from the horizontal 
displacement counter 31, the value of dy from the 
vertical displacement counter 32, and the value of sx and 
sy from the corresponding pel determinator 48 so as to 
generate the address for the pel (sx+dx, sy+dy) to be 
stored in the frame memory. Readout data is used to 
generate the value of a half-pel through the half-pel 



interpolator 232, if necessary, as shown in step S48 in 
Fig. 17. 

4) Prediction Error Power Calculation 

First, the prediction error power D(dx, dy) produced 
when dx and dy are expressed in terms of the motion 
vector is initialized to zero- This operation 
corresponds to step S45 in Fig. 17. The absolute value 
for a difference between the pel value read out in 3) and 
the value of the corresponding pel in the input picture 
luminance block is accumulated to D(dx, dy) in step S50. 
This operation is repeated until the condition x = y = 
block_size (in this example, block_size =8) holds in 
steps S52 and S54. Then, the prediction error power 
D(dx, dy) produced when the motion vector indicates the 
pel position (dx, dy) can be obtained. The pattern 
matching xanit 213 illustrated in Fig. 16 implements this 
operation. The pattern matching unit 213 supplies D(dx, 
dy) to the minimum prediction error power determinator 
216 through the prediction error power signal 215. These 
operations correspond to steps S49 through S54 in Fig. 
17. 

5) Minimiam Prediction Error Power Updating 

Then, it is determined whether D(dx, dy) obtained in 
4 ) has produced the minimum prediction error power among 
all the searched results that have been obtained so far. 
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The minimum prediction error power determinator 216 
illustrated in Fig. 16 makes this determination. This 
operation corresponds to step S55 in Fig. 17. The 
determination process is the same as that in the first 
embodiment. The value of (dx, dy) at that time is 
retained as the possible motion vector. This updating 
process corresponds to step S56 in Fig. 17. 
6) Motion Vector Value Determination 

The above-mentioned operations from 2) through 5) 
are repeated for all the search pel points (dx, dy) 
within the transformed block search range as indicated by 
steps from S57 through S 60 in Fig. 17. The finally 
retained dx and dy within the minimum prediction error 
power determinator 216 are output as the motion vector 
43. 

As described above, a prediction picture portion 
most similar t6 an input picture block with the minimum 
prediction error power produced is searched by using 
transformed block matching. As the result of the 
prediction picture portion search, the displacement from 
the predetermined starting pel of the selected prediction 
picture portion is obtained and expressed in terms of the 
motion vector 43. The prediction error power D_DEF 44 at 
that time is also retained. 

The above-mentioned motion vector 43 and the 
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prediction error power D_DEF 44 are used to determine the 
final motion compensated mode. This determination 
process is the same as that in the first embodiment • 

Next, the motion compensation process will be 
described. 

The motion compensator 9 implements the motion 
compensation process. In this embodiment, the operation 
of the corresponding pel determinator 37 is different 
from that in the first embodiment. Consequently, a 
description will be directed only to the operation of the 
corresponding pel determinator 37. The overall motion 
compensation operational flow is illustrated in the form 
of the flow chart in Fig. 8. 

In this embodiment, the corresponding pel 
determination is made as follows: 

When the motion compensated prediction mode 
represented by the motion parameters 11 indicates block 
matching, the corresponding pels are expressed with the 
sample points of a picture portion to which the 
displacement indicated by the input picture block 
position indication signal 206 is added and then 
translated to the area specified by the motion vector. 
This process corresponds to step S204 in Fig. 31, and 
represents an operation for determining the position of 
the pel (x+dx, y+dy) within the reference picture 28. when 
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dx and dy are expressed in terms of the motion vector. 

When the motion compensated prediction mode 
represented by motion parameters 11 indicates transformed 
block matching, the corresponding pels are expressed with 
the sample points of a picture portion to which the 
displacement indicated by the input picture block 
position indication signal 206 is added and then 
translated to the area specified by the motion vector. 
This operation corresponds to step S47 in Fig. 17 and 
represents an operation for determining the position of 
the pel (sx+dx, sy+dy) within the reference picture 28 
when dx and dy are expressed in terms of the motion 
vector. The prediction picture portion data readout 
operation and prediction picture portion generation are 
performed as in the first embodiment. 

A transformed block according to any one of the 
above-mentioned embodiments may have any arbitrary shape 
according to the following two assumptions: 

1) One-to-one correspondence is established between 
the pels of an input picture block and the pels of a 
prediction picture portion. 

2) Corresponding pels of the prediction picture 
portion in the reference picture are integer pels. 

A transformed block may have such a shape as tHat shown 
in Fig. 18 or Fig. 19. In addition, prediction picture 
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portions may be reduced or enlarged at an arbitrary ratio 
so as to be transformed into various shapes to allow 
block matching. By defining various shapes of the 
prediction picture portion in advance in this way, 
transformed block matching which gives the best match can 
be selected. In this case, the selected transformed 
block data is represented by the motion parameters 11 and 
is supplied to the entropy encoder 18. 

According to the above-mentioned embodiments, if 
only half-pels are interpolated, motion compensation for 
various types of motion including rotation and scaling 
down of an object can be performed. Furthermore, complex 
arithmetic operations whiqh are required when using the 
affine motion model are not needed for this invention. 
Thus, appropriate motion compensated prediction can be 
implemented even for the picture portion where the 
prediction error cannot be minimized merely by using the 
motion vector representing a translational amount, or for 
which correct prediction cannot be performed. 

In the embodiments described hereinbefore, a 
description has been directed to the case where integer 
pels or half-pels are used as the predetermined fixed 
pels of a transformed block. Other pels interposed 
between the integer pels at a 1 : 3 ratio, for example, may 
also be used as the fixed pels. In this case as well. 
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pel interpolation is not needed in the transformed block 
matching process, which is different from the 
conventional compensated prediction using the affine 
motion model. Thus, the number of steps needed for the 
process can be reduced, and high-speed processing can be 
performed. 
. Embodiment 5 . 

In the embodiments described hereinbefore, 
determination of block transformation parameters for each 
pel or the process for expressing the position of each 
pel in terms of the coordinates therefor is performed. 
Alternatively, corresponding pel determination may be 
made by preparing in advance a transformation pattern 
table such as a ROM storing the corresponding coordinates 
for the respective pels and determining the respective 
pels of a prediction picture portion based on the 
coordinates extracted from the transformation pattern 
table. Thus, transformed block matching and motion 
compensation having an arbitrary correspondence 
relationship between the pels of an input picture block 
and the pels of a prediction picture portion can be 
efficiently performed. 

Now, the above-mentioned operation will be described 
in conjunction with the first embodiment. 

Fig. 20 is a block diagram showing another internal 
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configuration (corresponding pel determinator 34b) of the 
corresponding pel determinator 34 illustrated in Fig. 5 
which realizes another embodiiaent of the present 
invention. In this embodiment, corresponding pel 
determination can be made by storing rdx and rdy in the 
ROM and determining the pel (rx, ry) corresponding to the 
pel (x, y) using rdx and rdy fetched from the ROM. 
Consequently, incrementation or decrementation of the 
transformation parameters rdx and rdy as shown in step S8 
in Fig. 6 are not needed. For making corresponding pel 
determinations, the rotated angle counter 33 illustrated 
in Fig. 5 is also not needed in this embodiment. 
Instead, corresponding pel determination can be made by 
providing a ROM table (transformation pattern table 100) 
inside the corresponding pel determinator 34b, as shown 
in Fig. 20. Through the corresponding pel determinator 
34b, the transformation parameters rdx and rdy are 
fetched from the transformation pattern table 100 based 
on the respective values x and y of the pel within an 
input picture block. Then, the transformation parameters 
rdx and rdy are supplied to the adder 110 and added to 
the motion vector data so as to determine the 
corresponding pel values. Then, the determined 
corresponding pel value data is supplied to the memory 
readout address generator 35. This method can also be 
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applied to other embodiments described hereinbefore- 
Thus, if only a few ROM (transformation pattern table 
100) memories are added to the corresponding pel 
determinator circuit 34b, the structural element for 
performing arithmetic operations for determining 
corresponding pels is not necessary and need not be 
provided. In this way, the corresponding pel 
determinator circuit 34 can be simplified and the number 
of the corresponding pel determination processing steps 
can be reduced. In addition, corresponding pel 
determinations can be made for a transformed block as 
shown in Fig. 21 which cannot be simply expressed in 
terms of transformation parameters. Thus, a 
transformation pattern library storing more abundant 
transformation patterns can be conceived. 
Embodiment 6 . 

In this embodiment, an encoder is disclosed which 
can decrease variations in the frequency characteristic 
of a prediction picture portion that is separated from 
the reference picture as a transformed block and which 
can reduce mismatches when implementing prediction for an 
input picture block. 

When a prediction picture portion includes half -pels 
as well as integer pels, a variation in the spatial 
frequency characteristic occurs between the integer pels 
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and the half-pels • An input picture block includes only 
integer pels. Consequently, this variation in the 
spatial frecjuency characteristic can become a factor 
which may cause a prediction mismatch. In this 
embodiment, after a transformed block has been defined 
similarly as in the other embodiments described 
hereinbefore, filtering of integer pels will be 
performed. 

Interpolation of half-pels can be performed by 
filtering integer pels thereabout at a [1/2,1/2] ratio* 
In order to do so, a low-pass filter filtering only ecpial 
to or lower frequency component than the cos(Qt/2) is 
provided. Prediction picture portions defined in the 
embodiments described hereinbefore include both 
unfiltered integer pels and filtered half-pels generated 
by the above-mentioned filtering. Consequently, a 
variation in the spatial frequency characteristic will 
occur in the prediction picture portions. When 
prediction accuracy has been reduced due to this 
variation, the unfiltered integer pels should also be 
filtered by the filter having a filter characteristic 
that is similar to the filter characteristic described 
above. Thus, the prediction accuracy can be enhanced. 

Fig. 22 is a explanatory drawing showing an example 
of the above-mentioned filtering operation. Fig. 22 
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shows the case where a low-pass filter F is used to 
filter the integer pels at a [1/8,6/8,1/8] ratio as shown 
in the following equations (7) . 
l(x,y) = Fy[Fx[I(x,y)]] 

F[I(n)] = (I(n-l) +6*I(n) + I(n+l))/8 (7) 
This filter filters equal to or lower frequency 
component than {cos {G)t/2) }2 so as to reduce a deviation 
in the spatial frequency characteristic in a prediction 
picture portion. After the low-pass filtering, a one-to- 
one correspondence is established between the pels of an 
input picture block and a prediction picture portion. 
Then, transformed block matching, determination of the 
motion vector, and motion compensated prediction mode 
determination are made, as in the embodiments described 
hereinbefore . 

A filtering operation and associated configurations 
of the transformed block matching unit and the motion 
compensator for implementing the filtering operation will 
now be described. 

In this embodiment, the transformed block matching 
unit and the motion compensator have configurations which 
are different from those described in the above-mentioned 
embodiments. A transformed block according to this 
embodiment is simply a reduced transformed block 
described in the fourth embodiment. In this embodiment. 
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the transformed block matching unit is regarded as a 
variation of the transformed block matching unit 42 
within the motion detector 8c and is indicated by- 
reference numeral 42c. Furthermore, the motion 
compensator is also regarded as a variation of the motion 
compensator 9 and is indicated by reference numeral 9b. 

Fig. 23 is a combination of explanatory drawings 
which describe an operation outline of the transformed 
block matching unit 42c according to this embodiment. 
Fig. 24 is a block diagram showing a detailed internal 
configuration of the transformed block matching unit 42c, 
Fig, 25 is a flow chart showing the operations of the 
transformed block matching unit 42c according to this 
embodiment . 

In the drawings, elements or steps assigned the same 
reference numerals as those in the aforementioned 
drawings mean the same elements or operations. 

First, the operation of the transformed block 
matching unit 42c will be described. With regard to 
operations similar to those described in the fourth 
embodiment, the description about these operation will be 
omitted. 

1) Operation Outline 

A transformed block can be defined in quite the same 
way as in the fourth embodiment. This embodiment is 
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different from the fourth embodiment in that the integer 
pels of a prediction picture portion are filtered. As 
shown in Fig. 23, A pels are provided in the reference 
picture so as to be filtered. A transformed block is 
comprised of A and □ pels. 

1) Initial Settings (Transformed Block Search Range and 
Initial Value Setting) 

These operations are performed in the same way as 
those described in the fourth embodiment. 
3) Possible Prediction picture Portion Readout 
Operation 

First, the pel (sx, sy) corresponding to the pel (x, 
y) in an input picture block is determined in the same 
way as that described in the fourth embodiment. Next/ 
the reference picture pel (sx+dx, sy+dy) is fetched from 
the frame memory. Then, it is determined whether the pel 
(sx+dx, sy+dy) is an integer pel or a half-pel. In other 
words, it is determined whether sx+dx and sy+dy are both 
half-pel components. The corresponding pel determinator 
48 illustrated in Fig. 24 makes this determination. This 
operation corresponds to step S62 in Fig. 25. When the 
reference picture pel (sx+dx, sy+dy) has been determined 
to be a half-pel, half-pel is interpolated by the half- 
pel interpolater 232. When the reference picture pel 
(sx+dx, sy+dy) has been determined to be an integer pel. 
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a filter 50 implements integer-pel filtering as shovm in 
Fig, 22. This operation corresponds to step S62 in Fig. 
25. 

With regard to the following: 

4) Prediction Error Power Calculation; 

5) Minimum Prediction Error Power Updating; and 

6) Motion Vector Value Determination, 

these operations are implemented in the same way as the 
operations described in the fourth embodiment. 

Next, the motion compensation process will be 
described. 

The motion compensator 9b implements the motion 
compensation process. 

Fig. 26 is a block diagram showing an internal 
configuration of the motion compensator 9b. Fig. 27 is a 
flow chart showing the operations of the motion 
compensator 9b according to this embodiment . 

In this embodiment, the filter 50 is provided within 
the motion compensator 9 illustrated in Fig. 7. The 
corresponding pel determinator 37 implements the same 
operations as described in the fourth embodiment. When 
the motion compensated prediction mode represented by the 
motion parameters 11 indicates block matching,, the 
corresponding pel is expressed with a sample point of a 
reference picture . portion translated from the input 
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picture block position indication signal 206 to an area 
specified by the motion vector. This operation 
corresponds to step S204 in Fig. 31. The reference 
picture pel (x+dx, y+dy) when dx and dy are expressed in 
terms of the motion vector is thus determined. 

When the motion compensated prediction mode 
represented by the motion parameters 11 indicates 
transformed block matching, the reference picture pel 
(x+dx, y+dy) is expressed with a sample point included i] 
a reference picture portion to which a displacement 
indicated by the input picture block position indication 
signal 206 is added and then translated by an area 
specified by the motion vector. This operation 
corresponds to step S47 in Fig. 17. The reference, 
picture pel (sx+dx, sy+dy) within the reference picture 
28 when dx and dy are expressed in terms of the motion 
vector is thus determined. In either case, it is 
determined whether the reference picture pel is an 
integer pel or a half-pel. When the reference picture 
pel has been determined to be an integer pel, the 
filtering of the pel as shown in Fig. 22 will be 
implemented in the same way as that implemented by the 
transformed block matching unit for generating a 
prediction picture portion. This filtering is 
implemented by the filter. The prediction picture 
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portion data readout operation and prediction picture 
portion generation will be implemented in the same way as 
those described in the first embodiment. 

The transformed block matching unit 42c may 
implement the search for an unfiltered prediction picture 
portion or a prediction picture portion filtered by the 
filter F and supply the searched result to the motion 
compensated prediction mode determinator 22. 
Alternatively, the transformed block matching unit 42c 
may implement the search for an unfiltered prediction 
picture portion alone and then filter the unfiltered 
prediction picture portion by using the filter F so as to 
determine the prediction picture portion exhibiting the 
greatest prediction accuracy. 

When the transformed block matching unit 42c has a 
facility for turning the filter F ON or OFF, the filter 
ON/OFF information should be included in the motion 
parameters 11. 

According to this embodiment, only the filtering of 
integer pels can reduce a deviation in the spatial 
frequency characteristic of a prediction picture portion. 
Thus, appropriate motion compensated prediction can be 
performed even for an input picture block where only the 
motion vector indicating a translational displacement 
cannot minimize a prediction error, or for which correct 
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prediction cannot be performed. 
Embodiment 7 . 

Fig. 28 is a block diagram showing a configuration 
of a video decoder for decoding compression encoded 
digital picture data, decompressing the compression 
encoded digital picture data, and reproducing a picture 
using the motion compensated prediction method according 
to this embodiment. Herein, a video decoder for 
receiving compression encoded data (which will be 
referred to as a bitstream) 19. generated by the video 
encoder described in the first embodiment, decompressing 
the bitstream, and reproducing a complete picture will be 
described. 

Referring to Fig. 28, reference numeral 51 indicates 
an entropy decoder, reference niimeral 6 is a dequantizer, 
reference numeral 7 is an inverse orthogonal transformer, 
53 is a decoding adder, . 54 is a frame memory, and 56 
indicates a display controller. 

The decoder according to this embodiment is 
characterized in the configuration and operation of the 
motion compensator 9. The configuration and operation of 
each element other than the motion compensator 9 has been 
described, and a detailed explanation will be omitted. 
The motion compensator 9 is the same as that illustrated 
in Fig. 1. Consequently, an internal configuration of 
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the motion compensator 9 is as illustrated in Fig. 1, and 
an operation flow chart thereof is as illustrated in Fig. 
8. 

The operation of the motion compensator 9 having the 
above-mentioned configuration will be described. 

First/ the bitstream is analyzed through the entropy- 
decoder 51 and separated into a plurality of encoded 
data. A quantized orthogonal transforming coefficient 52 
is supplied to the dequantizer 6 and dequantized using a 
dequantizing step parameter 17. The dequantized result 
is subjected to an inverse orthogonal transformation 
through the inverse orthogonal transformer 7 and supplied 
to the decoding adder 53. The inverse orthogonal 
transformer is the same as that within the encoder/ e.g./ 
DCT/ described hereinbefore. 

The motion parameters 11 decoded by the entropy 
decoder 51 are supplied to the motion compensator 9. 

The motion compensator 9 fetches the prediction 
picture data 12 from the reference picture data stored in 
the frame memory 54 according to the motion parameters 
11. According to the motion compensated prediction 
method of the present invention, a one-to-one 
correspondence is established between the pels of an 
input picture block and the pels of a prediction picture 
portion. Consequently, as in the motion compensated 
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prediction using the conventional block matching method, 
a single prediction picture portion is determined by 
using the motion parameters 11. 

Based on the value of the intra-/interf rame encoding 
indication flag 16, intraframe encoding or interframe 
encoding is selected. Then, when intraframe encoding has 
been selected, the decoding adder 53 outputs the output 
of the inverse orthogonal transformer as the decoded 
picture data 55. While on the other hand, when 
interframe encoding has been selected, the decoding adder 
53 adds the prediction picture data 12 to the output of 
the inverse orthogonal transformer and outputs the added 
result as the decoded picture data 55. The decoded 
picture data 55 is supplied to the display controller 56 
to be displayed on a display device (not shown) . In 
addition, the decoded picture data 55 is also written 
into the frame memory 54 to be used as reference picture 
data for the subsequent frame decoding process. 

A motion compensator in a decoder of the present 
invention performs the same operations as that in an 
encoder described in the second through sixth embodiments 
described hereinbefore. Consequently, the same motion 
compensator can be applied to a decoder for decoding 
compression encoded picture data generated by an encoder 
according to the above-mentioned embodiments. Half -pels 
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. as well as integer pels may be included in a predictive 
picture region and used, as in the video encoder • 

A video encoder and a video decoder of the present 
invention can be combined to configure a characteristic 
video encoding and decoding system. 

Further, by executing the operations illustrated in 
the flow charts described hereinbefore, that is, by 
providing a transformed block matching step, a 
corresponding pel determination step, a motion 
compensated prediction picture generating step, and a 
decoding adding step, a characteristic video encoding and 
decoding method can be obtained. 

Industrial Applicability 

As described above, according to the present 
invention, real integer sample point pels and half-pels 
interposed midway therebetween in a transformed 
prediction picture portion obtained by memory addressing 
alone are used to perform motion compensated prediction. 
Consequently, efficient and accurate motion compensated 
prediction can be performed for a picture portion in 
which correct motion compensated prediction cannot be 
performed by only using a motion vector limited to 
translational displacement. Further, this prediction can 
be implemented without performing complex arithmetic 
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operations which are required when using the af f ine 
motion model. Motion compensated prediction using a 
transformed picture portion for various types of object 
motion which cannot be expressed in terms of a numeral 
equation as well as object rotation and scaling which are 
easily expressible in terms of a numeral equation can 
also be performed by the invention- A video decoder of 
the present invention can also recreate a high-quality 
picture efficiently. 

Further, if only the memory addressing based on the 
corresponding pel determination is performed, various 
object motion including rotation, scaling down, or 
scaling up can be correctly predicted without performing 
complex pel interpolation such as when the affine motion 

model is applied. 

Further, by using the motion vector representing a 
translational displacement of an object through block 
matching, a transformed block search range used in 
transformed block matching can be effectively reduced. 
Overall operation processes for motion compensated 
prediction can also be reduced. 

Further, if only the memory addressing is performed, 
various object motion including simply reducing or 
enlarging can be efficiently predicted without performing 
complex pel interpolation such as when the affine motion 
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model is applied. 

Further, corresponding pel determination can be made 
by using a transformation pattern table. Consequently, 
motion compensated prediction for object motion such as 
that required for using the affine motion model and 
represented by an arbitrary transformed prediction 
picture portion and which cannot.be expressed in terms of 
a simple numerical equation can be performed. 

Further, by using a filter, a variation in the 
spatial frequency characteristic within a trans foanaed 
block can be eliminated. Thus, a prediction mismatch can 
be reduced. 

According to the present invention, a video decoder 
combined with a video encoder capable of performing 
transformed block matching and motion compensated 
prediction is provided. Consequently, the video decoder 
of the present invention can decode encoded picture data 
obtained by implementing rapid and optimum motion 
compensated prediction, into a complete picture. 
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CLAIMS 

1. A video encoder for performing compression encoding 
of input video divided into predefined blocks, including 
a motion compensated prediction means for implementing a 
motion compensation by detecting a motion between frames 
of blocks of an input video r the video encoder 
comprising: 

a motion detector, including a transformed block 
matching unit for transforming a reference image portion 
comprised of only integer pels, which consist of real 
sampling points, into an image portion having a 
predefined shape, for determining addresses of pels of a 
transformed reference image portion, and for matching the 
transformed reference image portion with the block of 
input video comprised of integer pels, and the motion 
detector outputting a motion vector providing a minimum 
prediction error; and 

a motion compensator, including a corresponding pel 
de terminator for transforming the reference image portion 
based on a motion parameter obtained from a comparison 
result of the transformed block matching unit, and for 
determining pels corresponding to pels of the reference 
image portion, and outputting a prediction image portion. 

2. The video encoder according to claim 1, wherein the 
transformed block matching unit transforms a reference 



image portion comprised of half-pels as well as integer 
pels into an image portion having a predefined shape. 

3. The video encoder according to claim 1 or 2, further 
comprising a preprocessor for separating an input video 
into video objects each of which is an encoding unit^ the 
video encoder dividing the video object into blocks / 
detecting a motion, and compensating a motion of a 
divided video object. 

4. The video encoder according to claim 1 or 2/ 
wherein the transformed block matching unit 

transforms the reference image portion comprised of half- 
pels as well as integer pels into one of an image portion 
scaled down at a specific ratio and an image portion 
scaled up at a specific ratio, determines the addresses 
of pels of a transformed reference image portion, and 
matches the transformed reference image portion with the 
input image portion; 

wherein the motion compensator transforms the 
reference image portion based on the motion parameter, 
determines addresses of pels of a transformed reference 
image portion, and outputs a motion prediction image. 
5. The video encoder according to claim 1 or 2, 
wherein the transformed block matching unit transforms 
the reference image portion comprised of integer pels and 
half-pels into an image portion rotated by a predefined 
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angle, determines addresses of pels of a transformed 
reference image portion, and matches the transformed 
reference image portion with the input image; and 

wherein the motion compensator transforms the 
reference image portion based on the motion parameter, 
determines addresses of pels of a transformed reference 
image portion, and outputs a motion prediction image. 

6. The video encoder according to claim 5, wherein the 
transformed block matching unit transforms a reference 
image portion comprised of integer pels and half-pels 
into an image portion rotated by one of angles of ±45-, 
±90", ±135% and ±180*. 

7. The video encoder according to claim 1, 
wherein the transformed block matching unit 

transforms the reference image portion, which has been 
translated within a search range of the reference image, 
into one of an image portion scaled up, an image portion 
scaled down, an image portion rotated by a predefined 
angle, determines addresses of pels of a transformed 
reference image portion, and matches the transformed 
reference image portion with the input image, and 

wherein the motion compensator transforms the 
reference image portion based on the motion parameter, 
determines addresses of pels of a transformed reference 
image portion, and outputs a motion prediction image. 
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8, The video encoder according to claim 1 or 2, 

wherein the transformed block matching unit has a 
transformation pattern table for transforming the 
reference image portion, and matches the reference image 
portion transformed according to the transformed value 
obtained from the transformation pattern table with the 
input image comprised of integer pels and half-pels, and 

wherein the motion compensator transforms the 
reference image portion based on the motion paramieter, 
determines addresses of pels of a transformed reference 
image portion, and outputs a motion prediction image. 
9* The video encoder according to claim 1 or 2, wherein 
the transformed block matching unit selectively filters 
specific pels of the reference image portion extracted 
for matching, and matches the transformed reference image 
portion with the input image portion. 

10. The video encoder according to claim 1 or 2, wherein 
the reference image used for detecting the motion is the 
freime which is forward or backward in time and stored in 
a frame memory for reference, and wherein the transformed 
block matching unit matches the transformed reference 
image portion with the input image using the frame. 
11. A video decoder for decoding an input compressed 
video data, the video decoder including motion 
compensated prediction means for implementing motion 
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compensation by detecting a motion between frames/ 

wherein the motion compensated prediction means has 
a mechanism for determining pels of a corresponding image 
portion comprised of predefined integer pels by 
transforming the corresponding reference image portion 
into a predefined shape according to motion parameters 
included in the input compressed video data, for. 
extracting the integer pels determined, and for adding 
the transformed reference image portion and the decoded 
video signal so as to produce a decoded video data. 

12. The video decoder according to claim 11, wherein the 
motion compensated prediction means further includes a 
mechanism for determining pels of the corresponding 
reference image portion comprised of half-pels as well as 
integer pels by transforming the corresponding reference 
image portion into one of an image portion scaled up, an 
image portion scaled down, and an image portion rotated 
in the same way as a transformed block matching \anit of a 
motion detector comprised in a video encoder 
corresponding to the video decoder. 

13. A video encoding method for performing compression 
encoding of an input digital video, comprising: 

a motion compensated prediction means for storing a 
reference image, for dividing the reference image into 
predefined blocks, and for detecting a motion between 
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frames; 

transformed block matching step for transforming a 
reference image portion comprised of integer pels into an 
image portion having predefined shape, for determining 
addresses of pels of a transformed reference image 
portion to generate a prediction image portion, and for 
matching the transformed reference image portion with the 
block of the input digital video; 

corresponding pel determining step for determining 
pels of the reference image portion by using the address 
of transformed reference image portion based on a motion 
vector providing a miinimum error selected by the 
transformed block matching step, and for supplying a 
motion compensated output ♦ 

14. The video encoding method according to claim 13, 
wherein the transformed block matching step transforms a 
reference image portion comprised of half-pels as well as 
integer pels into an image portion having a predefined 
shape as a reference, determines addresses of pels of a 
transformed reference image portion to generate a 
prediction image portion, and matches the transformed 
reference image portion with the input digital video. 
15. The video encoding method according to claim 13 or 
14, 

wherein the transformed block matching step further 
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comprises a transformation pattern table, transforms the 
reference image portion based on transformation value 
corresponding to an address from the transformation 
pattern table, and matches the transformed reference 
image portion with the input digital video. 
16. A video decoding method for decoding an input 
compressed video data including a motion compensated 
prediction means for storing a reference image, for 
dividing the reference image into predefined blocks, and 
for implementing a motion compensation between. frames, 
comprising; 

a motion compensated prediction step for 
transforming a reference image portion comprised of 
predefined integer pels into an image portion having a 
predefined shape, which corresponds to a video encoding 
method of a transmitter, based on a reference parameter 
included in the input compressed video data, for 
extracting addresses of pels of a transformed reference 
image portion, and for outputting a prediction image 
portion; and 

decoding adding step for adding the prediction image 
portion, and for obtaining a reproduced video data. 
17.. The video decoding method according to claim 16, 
wherein the motion compensated prediction step transforms 
a reference image portion comprised of half-pels as well 
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as integer pels into an image portion having a predefined 
shape, extracts addresses of pels of a transformed 
reference image portion, and outputs a prediction image 
portion. 

18. A video encoding/decoding system comprising: 

a video encoder for performing compression encoding 
of input video divided into predefined blocks, including 
a motion compensated prediction means for implementing a 
motion compensation by detecting a motion between frames 
of blocks of an input video, the video encoder 
comprising: 

a motion detector, including a transformed 
block matching unit for transforming a reference 
image portion comprised of only integer pels, which 
consist of real sampling points, into an image 
portion having a predefined shape, for determining 
addresses of pels of a transformed reference image 
portion, and for matching the transformed reference 
image portion with the block of input video 
comprised of integer pels, and the motion detector 
outputting a motion vector providing a minimum 
prediction error; and 

a motion compensator, including a corresponding 
pel determinator for transforming the reference 
image portion based on a motion parameter obtained 
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from a comparison result of the transformed block 
matching unit, and for determining pels 
corresponding to pels of the reference image 
portion, and outputting a prediction image portion; 
and 

a video decoder for decoding an input compressed 
video data, the video decoder including motion 
conqjensated prediction means for implementing motion 
compensation by detecting a motion between frames, 

wherein the motion compensated prediction means 
has a mechanism for determining pels of a 
corresponding image portion comprised of predefined 
integer pels by transforming the corresponding 
reference image portion into a predefined shape 
according to motion parameters included in the input 
compressed video data, for extracting the integer 
pels determined, and for adding the transformed 
reference image portion and the decoded video signal 
so as to produce a decoded video data. 
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ABSTRACT 



It is an object of the invention to obtain a video 
encoder and a video decoder performing a highly accurate 
prediction of a picture, transformed by other than 
translation, within a short processing time in a small- 
scaled circuit. A video encoder for performing 
compression encoding of an input picture includes motion 
compensated prediction means for performing interframe 
motion detection* The motion compensated prediction 
means has a motion detector including a transformed blocJc 
matching unit for transforming a reference picture 
portion into a picture portion having a predefined shape 
for performing motion detection and a motion compensator 
including a corresponding pel determinator for 
determining integer pels of rea:i seimple points of the 
transformed reference picture portion corresponding to an 
integer pel of the input picture block according to the 
motion parameters supplied from the motion detector as a 
result of the comparison so as to supply prediction 
picture data. The video encoder of the present invention 
can be combined with a video decoder having a motion 
compensator which performs the same operations as those 
in the video encoder to configure a video encoding and 
decoding system. 
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Fig.6 
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Fig.14 
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Fig. 17 
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Fig. 25 
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Fig. 31 
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Fig. 37 
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Fig. 38 
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